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ABSTRACT 

The project purpose was to develop and field test a 
program of instruction for young deaf children which could be used in 
existing ciassrooms. The main program contained two areas of 
instruction in written language; verb vocabulary at a high level of 
generalization and sentence structure. The program materlgils were 
field tested with 78 children in the preparatory levels of, three 
schools. The only criterion for admission to the sample was a simple 
test of minimum word rcr-ognltlon ability. Subjects ranged from 5 to 
10 years of age, and from three and one-half to 10 1/2 years on 
psychometric scores. Three validation criteria, one final performance 
criterion and two measures of gain, were applied in 28 Instances to 
the pretest and posttest data from the sampie groups. The criteria 
were met in 21 instances, of the 78 children, 11 % achieved mastery in 
verb vocabulary, and 83% in sentences. Each of the sample groups 
evidenced statistically significant learning in all areas of 
instruction. The mean time required for the program was 4,7 hours. It 
was concluded that the program represented a very effective, as well 
as an efficient, method of teaching written language to young deaf 
children, (Author) 
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SUMMARY 



rhc cducatiorial rcLardation of deaf children is a source of >. Mi- 
st ant concern to their parents and teachers and to others interesi(;d 
in their future vocational and social adjustment. The present pi'Oject 
is one of many endeavors directed toward practical solutions for the 
problems involved in educating deaf children. The purpose of the pro- 
jecl was to develop and field test a program of instruction for young 
deaf children which could be used in existing classrooms. The goals 
of the Program were (1) to develop materials appropriate for children 
as soon as minimal word recognition skill has been acquired; ( 2 ) to 
provide instruction in content areas of written language which are dif- 
ficult to teach; and (3) to achieve a high level of generalization of ver- 
bal concepts. The specific goal was to bring the students to mastery 
of written language at the recognition lcvel--a sub-goal which leads to 
the acquisition of receptive and expressive language at the recall level. 
The terminal behavior selected to represent the achievement of the 
specific goal was a multiple -choice recognition response, specified 
by means of validation tests in each area of instruction. 

The main Program consisted of two areas of instruction in 
written language - -ve rb vocabulary at a high level of generaliaation 
and sentence structure. A third area, noun vocabulary, was pro- 
vided in a supplementary Program for students who needed instruction 
in the noun vocabulary prerequisite to the sentence portions of the 
main Program, For the purpose of administering the Program, a 
simple method of presentation, including an inexpensive reinforcing 
device, was developed. 

After extensive trial and revision of the Program materials, 
the final form was field tested with 78 children in the preparatory 
levels of two state residential schools and one private day school in 
the southeastern area. Level I at each of the three schools and 
Level II in one of the slate schools comprised the four sample groups. 
Within these groups a satisfactory score on a simple test of minimum 
word recognition ability was the only criterion for admission to the 
sample. The subjects ranged from five years of age at Le's'e! I to ton 
years at Level II and from three and one -half to ten and one -half years 
in related psychometric age scores. 



During field testing two major problems involving materials 
arose. Because of delays in obtaining printed materials, student use 



of the Program was interrupted at various points. In the initial sets 
of the Program an unsatisfactory procedure, used in a portion of the 
frames, led to confusion for many children, A solution to this probiem 
was found, and materials for later sets of the Program were modified 
to accommodate the new procedure. 

The effectiveness of the Program was evaluated by vaiidalion 
pretest and posttest measures in each of the three areas of instiuiction. 
Three validation criteria- -one final performance criterion and twm 
measures of gain- -were applied to the test results for each of the four 
sample groups. The final performance criterion selected w'as 80/90-- 
80% of the validat' ^ subjects achieving a ma;dery score of 90% on the 
Program posttcsti. Meeting this standard was the only aim for the 
supplementary Program in nouns. The McGuigan gain ratio compares 
the actual gain with the possible gain. Blake's measure of gain is a 
more rigorous standard which penalizes the programmer for students' 
preknowledge of the program content. 

With one exception, the Level Tj sample group met the criteria 
in all areas of instruction. Because of high pretest scores in sen- 
tences, Blake's criterion was missed by a very narrow margin. The 
McGuigan gain ratio was exceeded by all four groups in both areas in 
which it was applied. 

The notable exceptions to acceptable performance occurred with 
respect to the mastery criterion and Blake's criterion for the three 
Level I sample groups. In the area of verb vocabulary the mastery 
criterion was met by only two groups; however, Blake's criterion was 
attained by all three groups. For sentences the mastery criterion was 
met by two groups and Blake's criterion by only one. Two of the three 
groups met the 80/90 mastery criterion for nouns. 

The validation criteria were applied in 28 instances to the data 
from the four sample groups. The Program met the standards in 21 
instances. Of the total sample of 78 children, 77% achieved mastery 
in verb vocabulary, 81% in nouns, and 83% in sentences. Application 
of the sign test to the pretest and posttest scores showed significant 
learning by each of the four sample groups in all areas of instruction. 

For the main Program the estimated average time for the 29 
Level II children was 224 minutes; the 49 Level I children spent approx- 
imately 317 minutes. For the 47 children who took the entire supple- 
mentary Program the average estimated time was 63 minutes. The 11 
children who needed only a portion of this Program averaged approx- 
imately 28 minutes. Additional time required for related activities, 
such as distribution of materials in the classroom and testing, was 
estimated at 210 minutes per child. It is estimated that the slowest 
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child spent approximately 14 hours in all ftogram activities, 

From these results it was concluded that, although the valida- 
tion criteria were not met in all instances, nonetheless the Program 
potentially represents a very effective as well as an efficient method 
of teaching written language to young deaf children. 

It was recommended that, with the correction of the two major 
problems associated with the first field testing, the Program be field 
tested again. Further revision was recommended if the Program 
should fail to meet the specified criteria in the second field testing. 
Upon the attainment of rigorous validation standards, it was recom- 
mended that the Program be disseminated to schools for the deaf and 
that additional materials utilizing the same methods he developed. 
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INTRODUCTION 



The educational retardation of deaf children is a source of con- 
stant concern to their parents and teachers and to others interested in 
their future vocational and social adjustment. However, the recent 
surge of activity directed toward practical solutions for the educational 
problems of deaf children allows us to be optimistic regarding the 
future. "The U, S. Office of Education is committed to improving the 
education of handicapped children. Such a commitment requires a 
major emphasis on the development, field testing and dissemination 
of instructional materials " (7). The present project is one of many 
endeavors emerging from this commitment. Programmed instruction 
was chosen as the area for exploration and demonstration. 



(1.10) Relevant Background 

In recent years programmed instruction has become widely ac- 
cepted in general education and in industrial training. At the time the 
present project was conceived, the literature regarding this subject 
with the deaf was limited but nonetheless indicated that programming 
could be used effectively in teaching language to deaf child 'en.^ Fehr 
(4) programmed the indefinite article and Falconer ^3) aeve oped a pro- 
gram for teaching sight reading vocabulary to deaf children. In both 
studies it was demonstrated that learning had taken place. In compar- 
ing two groups of deaf children, one which received teacher directed 
instruction and the other which received programmed instruction, 
Stuckless and Birch (12) found no difference in the amount of learning 
by the two student groups. However, the teacher directed group con- 
sumed twice as much time. 



(1.20) Characteristics of Programmed Instruction 

It has been demonstrated (6) that behavior can be shaped by im- 
mediate reinforcement of responses which approach desired goals -- 
that by introducing successive directional increments toward those 
goals, one can lead a student to a specified type and level of behavior. 
Programmed instruction can implement principles of learning associ- 
ated with the shaping of behavior (2) which are difficult to apply in a 



conventional classroom situation. It provides a setting for the fol- 
lowing: 

1, Immediate reinforcement 

2, Active responses from the student 

3, Reduction in learning errors 

4, Logical sequencing of learning steps 

5, Individual learning pace 

6, Efficient use of time 

A well designed program provides a sequence of learning steps 
which lead the student from the basal level (the level at which he can 
already perform) to a clearly specified goal (terminal behavior). At 
each step a stimulus is presented, a response is required from the 
student and immediate feedback (reinforcement) for the response is 
supplied without delays between components of the step. Further- 
more, the student can proceed from one learning step to the next 
through an entire sequence at his own rate of speed. With the cor- 
rection of errors at each step provided by feedback, materials can be 
logically sequenced--the accomplishmeid of one step providing the 
base for the next one. In this way independent activities can be uti- 
lized in the teaching of new material- -not for drill only. Correction 
of errors can prevent the practice of errors but does not prevent their 
occurrence the first time. Carefully designed learning steps and con- 
trolled sequencing of steps can greatly reduce the occurrence of 
errors which lead to erroneous associations. 



(1, 30) Characteristics of Conventional Methods 

In the conventional classroom setting it is difficult for the 
teacher to apply such learning principles. For many group activities 
the teacher simply presents a series of stimuli without requiring overt 
responses and without providing reinforcement. In group presentations 
in which responses are required and reinforcement provided, the 
teacher must deal with the children one at a time at some point in the 
lekrning step. As a result delays occur between the stimulus and the 
response or between the response and feedback. In either case, de- 
lays which interfere with the shaping of behavior are introduced at 
some point in the learning step. The time spent in waiting for others 
to perform is wasted and frequently attention is lost. Furthermore, 
in any Bituatioh which stimuli are presented to the group as a whole, 
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the p' ce of instruction is not likely to be appropriate for all members 
of the group. If the level of difficulty is geared to the slowest child, 
the pace is wasteful for all others. 

In many group settings ,* the learning activity is actually con- 
ducted on an individual basis. As each child takes his turn, the 
teacher presents a stimulus, requires a response from him and sup- 
plies reinforcement. Although there may be no delays within the 
learning step, the waiting time is even greater than with group pre- 
sentation of stimuli. 

In individual activities, commonly known as seatwork, the 
child may progress through a series of items at his own rate with- 
out delay between stimulus and response. However, there is usually 
long delay between response and feedback while the child waits for 
his teacher to correct his paper. This delay not only interferes with 
the sequence of events in a learning step; it also allows the child to 
practice his mistakes . Furthermore, design of sequential materials, 
when correction does not occur at each step, is difficult because one 
cannot assume the accomplishment of previous steps in designing 
subsequent ones. 



(1.40) Combining Programmed Instruction with Teacher Directed 
Activities 

It is not possible at the present time to estimate the amount of 
time the deaf child in a typical classroom situation spends waiting 
or to estimate the extent of interference with learning caused by de- 
lays. We do not suggest that this waiting time could be completely 
eliminated in the near future or that all group activities should be 
abandoned even if this were possible. However, we feel that the 
waste of student time could be substantially reduced and teaching 
effectiveness greatly increased through the combined use of pro- 
grammed instruction and teacher directed activities. 

Although a great deal of the deaf child's learning is necessarily 
subject to the personal control of the teacher, many of the conventional 
learning activities could be replaced by programmed instruction, thus 
providing individual attention without the loss cf time spent waiting. 

In addition, a greater degree of self-reliance could be developed through 
the reduction in the amount of teacher support necessary. Activities re- 
quiring the supervision of the teacher could be conducted in smaller, 
more homogeneous groups so that children would spend less time wait- 
ing while others perform, and the pace of the smaller group would be 
more compatible with that of the individual members. In the total 
learning situation a far greater number of learning responses could be 
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elicited and the teacher's professional skill could be utilized more ef- 
fectively. 



(1, 50) Advances in Educational Media 

It is encouraging to note that recent advances in educational 
media are being brought to bear on the problems involved in educati; 
deaf children- -both in programmed instruction and in teacher directed 
instruction. As with advances in any field, there are problems aris- 
ing which impede the widespread application of newly developed media 
especially that which is complex and costly. In addition to the expense 
of purchasing equipment and the accompanying materials, few existing 
classrooms have the space or wiring to accommodate the equipment 
necessary. It is hoped that the development of effective instructional 
materials in all types of media for the deaf will continue at an even 
more rapid rate. While the use of complex equipm nt may be essen- 
tial in some areas, it would appear that the applic ition of the princi- 
ples of modern learning theory through simple and inexpensive pro- 
gram devices could facilitate learning for deaf children. Certainly 
development of such methods and materials should not be neglected. 

In order to explore the utilization of inexpensive media in pro- 
gramming abstract language terms (classificatory nouns) for deaf 
children, a pilot study (5) was conducted at the Atlanta Speech School, 
The program developed for this purpose was presented in a pro- 
grammed text with accompanying answer sheets. It was adminis- 
tered to a small group of deaf children between the ages of seven and 
nine and to a similar group of normal children. The educational re- 
tardation of the deaf children demonstrated on a pretest was no longer 
evidenced on the posttest. Although programmed instruction appeared 
to have potential as an effective and efficient method of teac' ing lan- 
guage to deaf children, programs availabie for classroom use at the 
time the present project was initiated were limited in number and 
scope. 



(1.60) Des cription of the Project 

(1,61) Purpose and Goals. On the basis of results of the pilot 
study, it was decided to explore more extensively the application of 
programmed instruction to the teaching of written language for deaf 
children. The specific purpose of the project was to develop and field 
test a program of vocabulary development and sentence structure. 

The goals of the Program were as follows: 

1. To devise materials appropriate for young deaf children with 
minimal word recognition skill 
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2, To provide instruction in content areas of written language 
which are difficult to t<sach 

3. To achieve a high level of generalization of verbal concepts 

With regard to the third goal, it has been shown (8, 9, 13) that 
deaf children function at lower levels of generalization than do hear- 
ing children. However, the results of the pjdot study (5) indicate that 
this deficiency is amenable to teaching. Development of a high level 
of generalization was to be a major consideration throughout the 
development of the Program, A large number of examples would be 
needed for each element (shade of meaning), and elements must be 
carefully ordered to facilitate the perception of the common charac- 
teristic from one example to the next in the sequence. In order to 
prevent the association of the verbal symbol with a narrow concept-- 
or a completely erroneous one, a sequance of frames designed to 
develop generalization would be presented before the introduction of 
the printed symbol. Each of the various elements --demonstrated 
with different examples --would then be associated with the printed 



(1. 62) Criteria for Goal Achievement. At the time the Program 
goals were being formulated, language teaching was envisioned as con- 
sisting of three phases • 

1. Initial concept development 

2. Association of the verbal symbol with that concept at the 

recognition-response level 

3. Receptive and expressive use of the verbal symbol at the 

recall-response level 

It was not the aim of the Program to attain the third goal, lan- 
guage usage at the recall level, but to bring the student to mastery of 
the second phase- -a sub-goal leading toward the third goal. By pro- 
viding the instruction for the first two phases at a high level of gen- 
eralization, the Program enables the teacher to concentrate her time 
and energies toward instruction in the third phase. Furthermore, if 
the first two phases are thoroughly established, the development of the 
third phase is greatly facilitated, 

A great deal of consideration was given to the selection of termi- 
nal behavior--the behavior which serves as evidence that the instruc- 
tional goals have been attained. The type of behavior selected was a 
multiple -choice recognition response. In the construction of validation 
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tests utilizing this response mode, the level of generalization was 
measured by including items for all elements taught for each word. 

In addition, specific examples used in the test were different from 
those used in the Program frames. 

The level of behavior selecte 1 as the goal for the Program stu- 
dents was 90% mastery rather than the traditional 70% or 75% passing 
level. The rational for this criterion is two-fold. First, the alloca- 
tion of student time for instruction assumes that the content of that 
instruction is worthwhile. Second, almost universally the learning of 
young children serves as the foundation for future ler ing. If the 
foundation at any point is not thoroughly established, aie effect of an 
inadequate background is compounded in subsequent learning. 

(1.63) Requirements for the Method of Presentation. Physical 
requirements for the method of presenting the Program were established 
as follows: 

1, The method of presentation must first fulfill the require- 
ments for administering the teaching sequence effectively and 
efficiently. It must be flexible enough to accommodate any 
variation in frame format which might be needed. 

2, The device must be simple for students to operate and for 
teachers to maintain. The possibility of jamming must be 
virtually eliminated, 

3, Both the original purchase price of the device and main- 
tenance costs must be low in comparison to most teaching 
machines. It must permit the presentation of reusable ma- 
terials , 



4, The students must be able to use the Program in their usual 
setting for individual work and classroom storage requirements 
must be minimal. The device must house its own power supply, 
if required, 

(1.64) DeveTopment of the Program , After the formulation of 
specific instructional goals and presentation requirements, the follow- 
ing steps remained: (1) design of a model lesson plan, (2) construction 

of trial materials, (3) trial and revision of materials, and (4) develop- 
ment of a suitable method of presentation. When, after extensive trial 
and revision, a program of instruction appears to have reached an 
acceptable level of effectiveness. It must be field tested with an appro- 
priate s ample . 
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PROCEDURE 



In order for the Program to have the best potential for early 
and lasting effects upon education, it was decided that the target popu- 
lation would be children in the primary levels of schools for the deaf. 
The overall development of the Program included: 

(2. 10) Selection of subjects 

(2.20) S election of content 

(2. 30) Specification of terminal behavior 

(2.40) Design and preparation of materials 

(2, 50) Trial and revision of materials 

(2. 60) Determination of initial behavior 

(2, 70) Development of the teaching machine 

(2, 80) Collection of datr for validation 



(2. 10) Selection of Subjects 

Children in the preparatory levels in two state residential school 
and one private day school in Georgia and North Carolina were used in 
the validation phase of the project; only children in one state school 
were used during the trial and revision phase, 

(2. 11) Trial and Revision Subjects, Small groups of children 
were needed from time to time for the purpose of testing and revising 
material as it was being developed. The supervisors were asked to 
select children who, in their judgment, were representative of the 
range of achievement to be found in the particular level for which ma- 
terials were being developed, 

(2.12) Validation Subjects. When a program appears to have 
reached an acceptable level of effectiveness, it must be tested under 
appropriate classroom conditions for the purpose of validation; thus, 
when extensive trial and revision appeared to have brought the project 



materials to an acceptable level, a sample for validation was required. 
Within the two state schools the academic levels containing the youngest 
children were selected. Comparable subjects were selected from the 
day school. In subsequent discussions of the subjects there will bo 
reference to children in Level I, Level II or Level III. The reader 
must be aware that these may not be comparable in age from school 
to school; for example, the day school admits children to nursery 
classes at three and one-half years of age, one state school admits 
children to Preparatory Level I at five years of age and the other ad- 
mits to Preparatory Level I at six years of age. 

There was a total of 78 children who met the admission criteria 
established for validation subjects, discusst i in detail in (E, 83) 
Selection of Subjects. They were distributed as follows: 63% Level I 

and 37% Level II; 54% male and 46% female; approximately 86% Cau- 
casian and 14% Negro. 



(2.20) Selection of Content 

As originally envisioned, instructional materials would have en- 
QQjrjpg,ssed two areas. The first area would consist of a nucleus of 
verb vocabulary, for which the concept can be demonstrated wiwh a 
picture alone, and simple sentence structure. These would provide a 
base for the future teaching of more difficult concepts which require 
the use of a sentence as well as a picture for demonstration of the con- 
cept. 

The second area would consist of verb vocabulary and language 
Structure which require a sentence in addition to pictures for the 
demonstration of the concept, e.g, , such verbs as have 9-iid is and 
language structure such as tense and negation. Trial materials for 
tense and negation were devised and tried to a limited extent, but it 
did not appear that a satisfactory level of effectiveness could be 
Achieved within the time limitations of the project. Therefore, it was 
decided to concentrate the major effort of the present project in the 
first area. 

The Program Unit devised for this area had as its primary 
goal teaching — -at a high level of generalization— —the verb vocabulary 
to be utilized in the development of sentence structure. An additional 
goal was the developinent of an understanding of the relationship be- 
tween the verb and other parts of the sentence. In order to develop the 
Prograiri Unit, a series of trials was conducted to identify a strategy 
for structuring content into a workable framework. As a result of 
these trials the Unit was organized into five subdivisions, hereafter 



referred to as Sets One, Two, Tliree, Four^' and supplementary Sub- 
ject Noun Program, Set One consisted of verb vocabulary and sen- 
tence sequences, which together with the supplementary Subject 
Noun Program, was presented in the following order: 

(2. El) Four verb vocabulary words, each including a variety 
of elements 

(2.22) A supplementary program of eleven subject nouns 

(2.23) A sentence structure sequence 

Sets Two, Three and Four contained the following: 

(2.21) Four verb vocabulary words, each including a variety 
of elements 

(2.23) A sentence structure sequence 

(2.24) Review 

(2.21) Verb Vocabulary, Tlie first consideration in selection 
of the verb vocabulary within a given Set was suitability for use in 
developing sentence structure. Verbs were selected whose actions 
could be performed by many of the same subjects and received by 
many of the same objects. Careful selection of both nouns and verbs 
permitted the presentation of many simple sentences with a relatively 
small vocabulary. 

The present progressive tense was elected for use in the sen- 
tence structure portions of the Unit; therefore, verbs chosen must lend 
themselves to presentation in that tense. In addition, verbs in the Pro- 
gram Unit were selected to provide the basic vocabulary for the later 
teaching of verb tense; therefore, attention was given to the distribu- 
tion of verbs, with regard to tense characteristics. There was a need 
to include regular verbs which retain the root form when tense is 
shifted and irregular verbs where the change in tense alters the root 
word but maintains a reicognizable form. 

The second consideration was the predicted familiarity with the 
concept of each verb. At least one element of each concept had to be 
within the presumed expejrience of all the children. The degree of 
competition among the verbs within each Set was also an important 
factor in vocabulary selection. It was important that the words be non- 
competitive with regard to both concept and configuration. Another 

*As will be explained in (2, 872) Classroom Participation, Set 
Four was not validated. 



aspect given attention was the ease with which each concept could be 
demonstrated with pictures. It should be noted that this is not 
synonymous with the degree of artistic skill required. 

(2.22) Subject Noun Vocabulary. For those children who did 
not have the prerequisite noun vocabulary for the sentence structure 
sequence, the supplementary Subject Noun Program was devised. 

These nouns were selected on the basis of their usefulness in per- 
mitting the presentation of all the verb elements in simple sentences 
with a relatively small vocabulary. The children's probable famili- 
arity with the concept of each noun was considered equally important. 
Some degree of competition among words was unavoidable but, since 
nouns are generally considered to be less difficult for deaf children 
than verbs, there was less concern regarding competition among 
nouns. Portraying the concept graphically was not a problem. 

(2.23) Se ntence Structure Sequence. As soon as minimal vocab- 
ulary had been taught, verbs were combined with nouns in simple sen- 
tences, In Set One, following the vocabulary portions of the Program, 
the relationship of the verb to the subject was developed. Following 
the vocabulary sectio"s of Sets Two and Three, subject-verb sentence 
structure was reviewed Incorporating the cumulative vocabulary. 

Later sections of Set Three provided instruction in the relationship of 
the verb to the object, using the total vocabulary of the first three Sets. 

Since single pictures of on-going activities were to be used, the 
present progressive was selected as the appropriate verb tense for 
sentence structure portions of the Program. 

(2.24) Review. Sets Two, Three and Four contained sections 
which reviewed previous material (vocabulary and sentence structure), 
integrating new material with that of the preceding sets. A detailed 
outline of the content of the Program Unit appears in Appendix A. 



(2. 30) Specification of Terminal Behavior 

Terminal behavior is the behavior expected of the student when 
he has completed a program- -the behavior which is accepted as evi- 
dence that the goals of the program have been attained. It is usually 
specified by means of a criterion or validation test. 

A multiple -choice recognition response was selected as the be- 
havior which would serve as evidence that the goals of the Program 
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Unit had been attained. Validation tests were developed for the fol- 
lowing: 

(2.31) Verb vocabulary 

(2. 32) Noun vocabulary 

(2. 33) Sentence structure 

(2, 31) Verb Vocabulary Tests. A 54-item test covering the verb 
content of the Unit of four Sets was constrvvted to serve as the primary 
validation instrument. Each item presenta the student with a picture 
portraying a certain concept; his task is to select' the appropriate word 
from among competing alternatives. Alternatives were drawn from the 
vocabulary content of the entire Unit. Generalization was a primary 
goal of the project* therefore, two steps were taken to assure its accom- 
plishment: 

1, To prevent the student from responding on the basis of rote 
memory of specific frames from the Program, pictures 
appearing in the test were not used in the Program, 

2. One item for each major element of every vocabulary word 
taught was included in the test. 

In addition to the 54-item verb test, a shorter progress test for 
each Set was constructed. The items in these tests were limited to 
the content of the respective Seta and served only as interim progress 
checks. The results were not used for the purpose of validation, 

(2, 32) Noun Vocabulary Tests. A 28-item test, following the 
principles applied in the construction of verb tests, was devised to 
measure achievemisnt for the subject noun segment of the Program. 

Like the verb tests, it contains several items for each of the eleven 
vocabulary words, and the pi tures used in test items do not appear 
in the Program. Alternatives include words competitive in both con- 
cept and configuration. 

(2, 33) Sentence Tests. There were three tests for sentences: 
the first tested understanding of the relationship of subject to verb and 
contained 20 items; the second tested understanding of the verb-object 
relationship and contained 20 items; the third covered the subject-verb- 
object relationship and contained 30 items. 

Two types of multiple-choice items wsre used to test understand- 
ing of the verb-sentence relationship- -one type required a word response 
and the other type a picture response. 
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The word-response items presented the student with a stimulus 
picture and an elliptical sentence. The stimulus picture contained 
two or more subjects, each performing a different action. The stu- 
dent’s task was one of the following: 

1. To read the subject of the incomplete sentence, locate that 
subject in the picture, and select the appropriate printed 
verb for the sentence blank 

2. To read the object of the sentence, locate that object in the 
picture, and select the appropriate printed verb for the sen- 
tence blank 

3. To read both the subject and the object of the sentence, 
locate the subject with the object, and select the appro- 
priate printed verb for the sentence blank 

The picture -response items presented the studer with a simple 
sentence. His task was to read (1) subject and verb or (2) subject, 
verb and object, in order to select the appropriate picture. 

The validation tests dejscribed above were administered as pre- 
tests and posttests in order to measure gain from the Program. As 
Will be explained in (2. 88) Validation Testing, only 40 of the 54 items 
in the Unit verb vocabulary test were scored. In addition, only the 
first sentence test (subject-verb relationship) was used with validation 
subjects. The schedule for administering validation tests and the 
teaching material of the Program appears in Appondix A. Sampie items 
from all three types of tests are presented in Appendix C. 

(2,40) Design and Preparet|lon of Materials 

The learning task for the present Program was defined as one of 
generalization with multiple discrimination. The tentative teaching 
model involved a systematic progression through four levels; 

(2.41) The basal level (the level at which the student can 
already perform) 

(2.42) Concept development 

(2.43) Word association 

(2.44) Level of mastery 
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(2,41) Basal Level , In order to design a teaching seque^ne 
leading to mastery, it is necessary first to determine the target stu- 
dent’s basal level of performance- -the level at which he can already 
perform. From observation of students' behavior during an explora- 
tory phase of the project, it was pr«Fdicted that this level wt>iiid be 
low. Since it was the goal of the project to reach children as early 
as possible, every effort was made to adjust the beginning level of the 
Program to the most elementary level possible. Nonetheless, iji 
order to establish a starting point, preparation of the first trial ma- 
terials was based on the assumption of minimal word recognition skill 
and the ability to relate simple illustrations of familiar concepts to 
reality. 



From the very outset of the project, the ability to demonstrate 
the skill of word recognition at a minimal level was considered a pre- 
requisite. This skill was assessed not for the purpose of utilizing a 
particular vocabulary, but rather to ascertain the presence or absence 
of the skill. Children who failed the word recognition test, discussed 
in detail in (2, 61) Word Recognition, were not considered candidates 
for the Program, No attempt was made to devise a procedure which 
would overcome this inadequacy since it did not seem possible to de- 
vise an effective teaching procedure which could be administered dur- 
ing the five day period allotted for training and orientation. Thus ma- 
terials were prepared on the basis that the child had already developed 
sufficient word recognition skill for the learning tasks of the Program, 

During the early trials of the Program it became evident that 
many of the trial subjects were failing due to an inability to relate pic- 
tures to experience. It was decided that the teaching of this skill 
would fall within the scope of the project. Unlike word recognition, it 
seemed possible to devise an effective teaching procedure to be admin- 
istered during the five day training and orientation period. Activities 
designed to meet this need are described in (2, 62) Association of Pic- 
tures with Reality, 

Since the basal level of performance was expected to be low, a 
highly structured method of presentation was required. The tentative 
frame design provided a large area for the stimulus (picture and/or 
word) at the top of the frame with the response and two alternatives 
(pictures and/or words) in a horizontal arrangement across the bottom. 
It was decided that beginning frames should require only gross discrim- 
ination anc that progressively finer discrimination would be required on 
subsequent frames. 

(2.42) Concept Development. The plan of the Program followed 
a policy of developing a concept through pictures rather extensively 
before introducing the printed word. This policy tends to prevent the 
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association of the printed symbol with a narrow or completely erroneous 
meaning. 

Concept formation was achieved by a series of picture matching 
frames, beginning with the element predicted to be the most familiar 
and progressing to less familiar ones. Beginning with the most familiar 
one, elenaents were ordered in such a manner as to facilitate expansion 
of the concept. Each successive example shared a feature with 0 <• pre- 
vious one so that the learner could perceive the commonality as the 
range of meaning for each concept expanded. As the Program progressed 
the concept was further defined by the judicious u,se of alternatives which 
increased in similarity to the correct choice, 

(2.43) Word Association. When the concept for a word was 
thought to be sufficiently established, the printed symbol was associ- 
3 _t 0 d with examples of the various elements of the concept, A word- 
matching technique was employed to induce the student to attend and 
respond to the printed word. Early frames required only gross dis- 
crimination, but closer attention to word form was required as alterna- 
tives became more similar in configuration to the correct choice, 

(2.44) Level of Mastery. When the word-matching sequence had 
reached a level of fine discrimination, the support supplied by matching 
was removed. In order to facilitate the transition from word matching 
to word recognition, early recognition frames were constructed witn 
alternatives dissimilar to the response. In the final revision, cues 
were also used for this purpose. Frames approaching the terminal 
level alternated picture *esponses with word responses, each requiring 
successively finer discrimination until the desired level of mastery was 
attained. The final frames reflected the expected terminal behavior 
w'ith regard to difficulty and extent of generalization. An excerpt from 
the final Program appears in Appendix B. 

The model provided the framework within which trial materials 
were cast for subsequent presentation to trial subjects. The first 
lesson plans were written for only two words in order to obtain the 
feedback needed to discover the inadequacies in the tentative model 
and allow for appropriate changes before investing extensive time in 
the total Program. 

As the first lesson plans evolved, attention was turned to the 
method of presenting the Program. Basic requirements for the method 
were established. A machine, adequate for trial and revision of ma- 
terials, was constructed, but there was a continuing effort during thi^ 
time to improve the method of presenting materials. A detailed account 
is provided in the section (2.70) Development of the Teaching Machine. 
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(2. 50) Trial and Revision 



In order to bring a program to an acceptable level of effective- 
ness and efficiency, repeated trials and revisions usually must b^ 
conducted. This phase of the project began as soon as materials i. 
the first two verbs had been prepared, and continued concurrently 
with subsequent materials preparation for a period of 18 months. 

From time to time small sampleis of deaf children, considered 
by their supervisors to be representative of their grade levels with 
regard to achievement, were selected. During the revision period 
for the first Set of materials, several groups were used successively 
for short periods of time. Revisions were made after each trial. 
During later stages two permanent sample groups were used. The 
second group received materials after revisions from trials with the 
first group. 

Trial materials were administered individually by means of a 
trial model teaching machine. An observer recorded Program 
responses and other aspects of behavior, e.g. , confidence -uncer- 
tainty, interest-boredom, random respond ^-deliberate response. 
Analysis of program erfrors, together with other reactions, provided 
the basis for revision, 

(2,51) Revisions in the Model, The revision period for the 
first Set of the Program was a critical one, for it was during this 
time that student performance revealed certain basic weaknesses in 
the Program and pointed the way to an improved model. On the basis 
of students' responses during the trials of Set One, the following re- 
visions were made in the model* 

1, Control of irrelevant variables. Many of the children were 
responding on the basis of perceptible characteristics -- 
such as size, shape, line direction, degrees of shading, 
amount of detail, etc. Considerable effort went into the 
equalization of these influences within any given frame, 

2, Elimination of confusion factors. Confusion resulted be- 
tween words being taught in the same Setj the solution was 
the avoidance of other words from the same Set as alterna- 
tives, until the word being taught was firmly associated 
with the concept. Words were then brought together in 
carefully controlled sequences, 

3, Designation of separate tracks. The slower students re- 
quired more exposure to individual verbs before contending 
with them In competition. In order to prevent needless 
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repetition for the more capable students, separate tracks 
were established. Immediately before words in a Set appear 
in competition, the Standard Track (designed for the slower 
student) was interrupted and the foregoing material was re- 
peated. When the slower student reached this point the 
second time, he continued to the terminal frames of the se- 
quence. The more capable student followed the Accelerated 
Track which did not repeat, 

4. Addition of writing frames. In order to utilize an additional 
sensory avenue and to require closer attention to word form., 
tracing and, copying frames were inserted after the printed 
word had been introduced. 

5. Insertion of intermediate frames. Where inter-frame incre- 
ments proved too large, intermediate frames were inserted. 

6. Introduction of mediating symbols. Many of the students 
failed to integrate subject and verb in the sentence structure 
portion of the Program. Small pictograph symbols were 
placed above the words in the sentences to facilitate integra- 
tion, These symbols were later incorporated into the vocabu- 
lary sections to promote transition from word matching to 
word recognition. They also proved to be effective in bring- 
ing words in a Set together, 

7. Addition of red underlining. After pictograph symbols were 
added to facilitate integration of subject and verb in the sen- 
tence structure portion, soine of the students failed to inte- 
grate these two when the symbols were dropped. At this 
point red underscoring was used to direct attention to both 
words. Red v/as also used at other points of transition to 
prevent the disregarding of stimulus components. 

8. Addition of arrow comprehension frames. In the word re- 
sponse frames of the sentence structure portions of the Pro- 
gram, arrows were essential to induce the student to locate 
the subject in the picture after reading it in the sentence. It 
was discovered that moat of the students did not understand 
the purpose of the arrows. At this point a sequence of fram.es 
for teaching the comprehension of demonstrative , arrows was 
inserted, 

(2. 52) Later Revisions, Since many of the basic weaknesses in 
the model were corrected as a result of trials with Set One, subsequent 
materials did not require such extensive revision. Problems were gen- 
erally those peculiar to the specific content of the sequence. An excerpt 
from the final Program appears in Appendix B. 



(2. 60) Determination of Initial Behavior 

From observation of students' behavior during the exploratory 
phaBe of the project, it had been predicted that the basal level of the 
target students' performance would be low. Subsequent experience 
during the trial and revision period proved this prediction to be justi- 
fied. 



After all adjustments had been made in the beginning level of 
the Program, two initial behaviors (abilities prerequisite to success- 
ful performance) were specified: 

(2.61) Minimum word recognition skill 

(2.62) Association of pictures with reality 

(2. 61) Word Recognition. As noted in a previous section mini- 
mum word recognition skill was considered an ability essential for the 
student to poeseas at the time he began the Program; therefore, those 
students who did not demonstrate minimum word recognition were not 
a part of the target population. 

Minimum word recognition for an individual student was defined 
as the ability to recognize four out of six wordE', A test was con- 
structed for each of the three schools participating in the validation 
of the Program. From a list of the first ten words taught in each 
school, six were selected as the content for the test and two additional 
ones were used as sample items. All ten words appeared as alterna- 
tives , 



Each item presented the student with a simple drawing illustrat- 
ing the concept element judged to be most familiar. His task was to 
select the correct word from among four choices. Although the word 
recognition tests were not designed to measure fine discrimination 
with respect to word configuration or concept, it was necessary to pre- 
vent the student from responding solely on the basis of familiarity with 
the correct word in the presence of unfamiliar ones. At least two words 
from the total list of ten were used as alternatives in each item so that 
the students could not merely choose the one familiar word, but every 
effort was made to minimize competition between the correct choice and 
the alternatives with regard to word form and concept represented. 

In administering the test, instructions were accomplished by 
the use of gestures. Precautions were taken to insure understanding 
of the procedure. The two sample items v sre checked, and any 
assistance necessary for the correct completion of these items were 
given before the children were allo wed to proceed with the six items 
of the test. 




30 



20 



(2, 62) Association of Pictures with Reality. Since still drawiiigs 
were used throughout the Program to portray the concepts, it was essen- 
tial that the target students be able to associate pictures with reality. 

In the very early trials of the Program, it became obvious that many of 
the students' failureswere resulting from an inability to mal<e this associ- 
ation. In most cases this deficiency did not appear to be a function of 
intelligence. 

It was apparent that many potential target students would be ex- 
cluded from the Program unless some solution to the problem v/ere 
found; therefore, it was decided to test the effects of a brief training 
session upon the performance of unsuccessful trial students. A set of 
materials, designed to teach the relationship between pictures and 
reality was adniinistered to these children. Subsequent improvement 
in their Program performance justified the additional instructional 
time and the materials were further developed to accommodate the 
varying levels of ability. 

The resulting Initial Behavior Testing-Training Sequence (IBT-TS), 
in its final form, served a dual purpose; (1) to evaluate the students 
with regard to initial behavior required to cope with the Program, and 
(2) to train those who had not achieved an appropriate level of skill. 

The sequence was administered individually with feedback provided by 
the instructor. The child's first response to an item (right or wrong) 
was recorded as his score. If he failed the item, he was then helped 
to understand it b^^ore proceeding to the next one. 

In order to adapt the IBT-TS to the different levels of ability, 
three parallel sets of material were developed--one set for each of 
three successive days. Children who performed successfully on Day 
One or on Day Two were exempted from further training except for 
the last verb section which Involved picture matching. Table I is a 
surniTiary description of the stimulus -response tasks required of the 
student on one day. 



Table I, Description of the Stimulus -Response Tasks for On;: 
Day of IBT-TS. 



NOUN VERB 



Stimulus 

Presented 


Subf’ect 

Response 


Stimulus 

Presented 


Subject 

Response 


Picture of 
object 


Select 

real 

object 


Action by 
experimenter 


Select 
picture of 
action 


Real object 


Select 

picture 


Picture of 
action 


Child 
p'^rforms 
the action 






Picture of 
action 


Select 
picture of 



similar 

action 



The sequence for Day One and for Day Two consisted of two major 
divisions : 

(2.621) Nouns --teaching the relationship between pictured ob- 
jects and real objects 

(2.622) Verbs --teaching the relationship between pictured 
actions and real actions. 

Day Three omitted the noun division. IBT-TS for Day One is presented 
in Appendix D, 

The format of the sequence was designed to accustom the students 
to the Program format. Pictures were presented on 8 1/2" by 11" 
pages bound in looseleaf fashion. Accompanying objects were placed 
on a rectangular mat to the student’s right of the notebook. 

(2. 621) Nouns , Since noun concepts are generally found to be 
easier for deaf children to learn than are verbs, the noun division was 
administered at the beginning of the sequence on Day One and on Day 
Two. A third day of training in this area was not considered necessary. 
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In each of the two noun sections the student was required to 
associate real objects with pictures of objects. The degree of simi- 
larity in appearance between the stimulus and the response ranged 
from almost identical to distinctly different. Since the sole aim of 
IBT-TS was the understanding of pictures in relation to reality, no 
miniature or rtificial objects were used. 

(2. 622) Verbs . The first two verb sections for each day re- 
quired the student not only to associate pictured events with real 
actions but also to associate an illustration of one element (or shade 
of meaning) with a live action involving a different element of the same 
verb. Varying degrees of generalization were required. In the two 
sections involving live actions, the stiident was required to make two 
responses - -representing two different elements --to the same stimu- 
lus. As in the noun division, association of pictures with reality was 
the sole aim of the verb division of IBT-TS, therefore, no pretending 
was used in actions either performed by the instructor or accepted 
from the student. In the third section both the stimulus and the re- 
sponse were pictured verb concepts — the format representing that of 
the concept formation frames of the Program. 

(2,70) Development of the Teaching Machine 

(2. 71) Initial Considerations. From the outset the goal of the 
project was to develop programmed instruction which utilized an inex- 
pensive presentation. While there are obvious advantages in costly 
media such as film strips, motion picture films and video tape, it 
was felt that the use of still pictures in programmed instruction should 
not be neglected. In addition to cost, practical aspects such as operat- 
ing space, storage space, electrical wiring requirements, maintenance 
and simplicity of operation for both student and teacher were considered. 

Experience with target students during an earlier phase of the 
project made the following requirements apparent: 

1 , A highly structured presentation which could be achieved 
only with a machine 

2, Frames large enough to accommodate clear pictures and 
still allow ample spacing between the stimulus and the 
choices 

3, A simple, direct method of responding and receiving 
reinforcement 
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The target student could not be expected to make a 
two-step response involving an intermediate symbol 
(e, g. , to select his answer, note the number or 
letter designation of his choice and find this designa- 
tion in another place). 

He could not be expected to compare his choice with 
the correct answer and decide whether he was right 
or wrong, (In many programs the student compares 
his answer with the correct answer to obtain reinforce- 
ment. ) 

4, A simple method of proceeding from one frame to the next. 

The course of events which led to an acceptable teaching machine 
beg^n vrith a search for an appropriate machine already in existence. 
None was found which met all the criteria established. 

(E,7i-) Trial Model Teaching Machine, The first trial model 
devised consisted of a box which housed a stack of 8 1/2'' by 11" cards 
in a nearl'' upright position with the first card exposed. A narrow 
mask, attached to the box with springs, covered the lower two inches 
of the exposed card and held the stack firmly against a supporting sur- 
face, The student looked at the stimulus at the top of the exposed 
card, selected his answer from a row of choices just above the mask, 
and inserted a stylus into a round opening in the mask directly below 
his choice. If his answer was correct, a bulb at the base of the box 
was lighted. If his answer was wrong, the bulb did not light. Power 
was supplied by a battery housed in the box and wired to the bulb and 
the stylus. 

When the student had completed a frame (one frame to a card) 
he removed the card and placed it in a compartment for used cards 
at the back of the box. When he removed one card, the next one was 
expos ed. 



The machine described above was used effectively for several 
months in the individual administration of trial programmed materials. 
The device was relatively inexpensive, reusable materials were poa- 
sible and there were no special electrical wiring requirements. In 
terms of presentation requirements, the device allowed ample frame 
space and provided a direct method of response. 

In terms of the original project goals, the disadvantages ren- 
dered the device unacceptable for ultimate classroom use. In many 
classrooms the individual student's desk would not be suitable and the 
amount of storage space required would not be available. Although 



the within-frame procedure was simple, the procedure for advancing 
from one frame to the next was difficult and allowed materials to 
become out of order. Efforts to improve the method of presentation 
led to the development of a device which retained the desirable 
features of the trial model but eliminated its major dis idvantagcs . 

(2.73) Final Model of the Teaching Machine. The final model 
illustrated on the following page consists of 8 l/E” by 11" books btuind 
with plastic cones and a completely separate reinforcing device which 
approximates the size and shape of a primary pencil. 
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PROGRAM PRESENTATION AND REINFORCING DEVICE 

Procedure: 

1, The child looks at the exposed frame. 

2, He looks at the stimulus (A), 

3, He selects his answer from the choices (B) available. 

4, He touches the tip of the reinforcing device (C) to the ink patch 
under his choice. If he touches the patch (D)- under the correct 
choice, the light (E) comes on. If he touches a patch (F)** under 
an incorrect choice the light does not come on. He then makes 
another choice, 

5, He turns the page and looks at the next frame. The lower corners 
(G) of the pages are cut “-odd numbered pages are cut deeper than 
even ones--1o facilitate page turning and prevent turning two pages 
at once. 

onductive ink is used under the correct choice, 

^^'Nonconductive ink is used under incorrect choices. 
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At various points in the Program, the student encounters frames 
'vhich require him to trace or copy. These pages are laminated and 
contain no ink patches. The student writes on the laminated nacre 
with a pencil containing a special lead which can later be removed. 

The reinforcer is a self-contained unit housing batteries, light 
bulb and other electronic components. This arrangement eliminates 
the need for a box, thereby solving the problems of storage and oper- 
ating space. With the box eliminated, frames can be bound with plastic 
cones so that advancing from one frame to the next involves simply 
turning the page of a book. The book form also eliminates the oppor- 
tunity for materials to become out of order. In addition, the format 
restrictions of the first machine arc eliminated permitting a large 
variety of frame designs. 

The validation phase of the project was used for testing the 
method of presentation in a practical classroom situation. As teachers 
and project staff observed problems with the handling of materials, 
modifications were made. The method of presentation in use at the 
close of the validation period satisfied the following previously established 
requirements : 

1. Simplicity of Operation. Manipulation of materials is easy 
for young children and takes up a minimum of the total instructional 
time. There is virtually no possibility of jamming. The method of 
response is direct and reinforcement is immediate. The procedure 
for changing batteries and bulbs in the reinforcing device is as simple 
as replacing these parts in a flashlight. In addition, maintenance in- 
cludes wiping off the writing on a few plastic pages per book after each 
student's use, 

2. Cost Factors, The cost of the reinforcing device, less than 
$8. 00, is low compared to most teaching machines. Maintenance costs 
(replacement of batteries and bulbs) are also low; these parts should not 
need replacing more than twice in a school year. Although the printing 
is more expensive than conventional printing, the materials are com- 
pletely reusable. 

3. Building Requirements. Any individual desk provides adequate 
space for using the Program. Classroom storage for books requires 
space for two books per child-“the one in use and the next one in the se- 
quence, Reinforcing devices can easily be kept in a small box. The 
power supply is housed in the reinforcing device itself; therefore, no 
source of electricity is required from the classroom. 
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(2, 80) Collection of Data for Validation 



(2. 81) Validation Procediare. The final step in the develop- 
ment of programmed instruction is validation- -it must be demon- 
strated that target students learn from the program in the pructical 
situation for which it is ultimately intended. In addition to effective- 
ness, the efficiency of the program must be stated. The following 
steps, generally considered to be necessary, were employed in tht’ 
validation of the project Program: 

1. A sample representative of the target population was 
s elected. 

2. The subjects were pretested to determine the level of 
prior knowledge of the Program content. 

3. The subjects used the Program under target conditions 
and the instructional time was recorded. 

4. The validation posttests were administered in order to 
determine the level of learning at the conclusion of the 
Program. Performance on these tests constitutes the 
behavior which serves as evidence of the learning 
achieved through use of the Program. 

In addition to the above procedure, it was originally intended 
to compare the learning by the validation subjects with that of a control 
group by means of a standardized achievement test. The development 
of this test is outlined in Appendix E, Because of uncontrollable de- 
lays which occurred in the printing of programmed materials, this 
aspect of the evaluation had to be abandoned. These problems are 
discussed in Appendix F. 

(2. 82) Schedule of Subject Events. The collection of validation 
data involved the following steps, listed below in chronological order. 

Word Recognition Test 

Subject Noun Test, pretest 

Unit Verb Vocabulary Test, pretest 

Initial Behavior Testing-Training Sequence 

Selected psychometric test items 
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6, Classroom use of the Program including recording of time 
Introduction to teaching materials 
Set One vocahulary 
Subject noun vocabulary 

*7. Subject Noun Test, posttest 

'•'S, Subject-Verb Sentence Test, pretest 

9. Classroom use of the Program resun I 
Set One sentence sections 
Set Two 
Set Three 

*10. Unit Verb Vocabulary Test, poattest 

*11. Subject-Verb Sentence Test, posttest 

*These items comprise the Program validation testing. 

Tests were administered individually by trained personnel with directions 
accomplished by demonstration and gesture. Children were urged to re- 
spond to every item. The first three tests in the schedule were admin- 
istered in the order of predicted difficulty. 

{2.83) Selection of Subjects. The procedure for selecting sub- 
jects was designed to identify children who had acquired word recog- 
nition skill at a minimal level but had not yet acquired knowledge of 
the Program content. Within the sample selected, it was necessary 
to identify those children who needed instruction in subject noun vocabu- 
lary. The sample for validation was drawn from the preparatory levels 
of two state residential schools and one private day school in the south- 
eastern area. The reader is reminded that the age of admission is dif- 
ferent in each of these schools. 

(2.831) Prior Knowledge of Program'. Content. Scores from the 
Unit Verb Vocabulary pretest were used to identify academic levels, 
within schools, in which there was insufficient margin to demonstrate 
a satisfactory gain. Levels in which half or more of the students 
scored 70 % or better were excluded from the validation sample. 

Level III in two schools and Level II in one school were excluded on 
this basis. Because of problems which arose with the printing of Pro- 
gram materials (see Appendix F), Levels II and III in the remaining 
school had to be excluded also. None of these children participated in 
the classroom use of the Program. The levels remaining, Level I 
from all three schools and Level II from one state school, contained 
115 children. 



(2.832) Word Recognition Skill, The Word Recognition Test 
was used to identify children within these levels who did not demon- 
strate word recognition ability at a minimal level. Those who failed 
more than two of the six items in the test, 22 of the 115 children, 
were excluded from the sample. It had been predicted that such 
children could not succeed in the Program; therefore, they were not 
considered a part of the target population. However, they were in- 
cluded in all validation activities in order to test this assumption. 

(2.833) Incomplete Data. Sixteen additional subjects were 
lost because of incomplete data due to absence during pretesting, 
other preliminary activities, or posttesting, 

(2, 834) Summary of the Sample. The validation sample was 
composed of 78 subjects who showed low pretest performance, who 
demonstrated word recognition ability at a minimal level, and who 
participated in all validation activities. The distribution of subjects 
appears in Table 11. 
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(2, 835) Supplementary Program. Instruction in noun vocabulary 
was provided for validation subjects who had not previously mastered 
the nouns prerequisite to the sentence portions of the Pr "fram. 

Scores "rom the Subject Noun pretest were used to exempt children 
from the supplementary Subject Noun Program or from one of the two 
sections. Students who missed two or less of the 28 items were ex- 
cused from the entire supplementary Program. Those who missed 
one or less within either of the two subtests were excused from tlie 
corresponding section of the Program. 

(2. 84) Subject Information. Ii^ order to obtain uniform naeasures 
of learning ability across schools, three psychometric items were 
administered: (1) the Seguin Form Board from the Grace Arthur 

Point Scale of Performance, Revised Form II, (2) Picture Identifica- 
tion and (3) Picture Association from the Hiskey-Nebraska Test of 
Learning Aptitude. Tests were chosen on the basis of their apparent 
relationship to the tasks required by the Program.. The Seguin Form 
Board is a test of visual-motor function which is not bound to social 
or verbal skills. The Picture Identification portion of the Hiskey in- 
volves the visual matching of identical items whereas the Picture 
Association requires more abstract associations. These were admin- 
istered during the three day period in which the Initial Behavior Test- 
ing-Training Sequence was administered. 

In addition to the psychometric data, audiometric data v/ere ob- 
tained for the children in two of the schools. The average hearing 
level for 250 and 500 H., can be found in Table II, 

(2, 85) Initial Behavior Testing. Initial behavior in two areas 
was evaluaLed, As previously discussed, inadequaLe performance on 
the Word Recognition Test excluded children from the validation sam- 
ple, The Initial Behavior Testing -Training Sequence was administered 
in order to assess the ability to associate pictures with reality; to pro- 
vide instruction for children who demonstrated a deficiency in this 
area; and to serve as an orientation to the Program format. The 
three parallel sessions were administered individually on three suc- 
cessive days, each session requiring approximately 15 minutes per 
child. 



(2. 86) Track Assignment. During the trial and revision phase, 
it became apparent that different Program tracks would be needed in 
order to provide sufficient exposure for some students and yet avoid 
needless repetition for others. The primary considerations in initial 
assignment to Standard or Accelerated Track were performance on 
IBT-TS and in the first three sections of the Program, In border- 
line cases consideration was also given to scores on psychometric 
test items. All children who failed the Word Recognition Test were 
assigned to the Standard Track, 



Subsequent reassignments were based primarily on scores from 
the progress tests following individual Sets. These tests served only 
as interim progress checks--the scores were not a part of the valida- 
tion data. If a student in the Accelerated Track missed two or more 
items on the progress test for one Set, he was assigned to the Standard 
Track for the next Set. If a student in the Standard Track achieved a 
perfect score, assignment to the Accelerated Track for the next Set 
was considered. Such decisions were less objective because judgments 
were necessary regarding the child's ability to maintai’i mastery per- 
formance without the repetition provided by the Standard Track. In 
such cases judgments by the child's teacher and the project supervisor 
were the determining factor. 

(2. 87) Classroom Use of the Program. Since the Program was 
designed to be used within the framework of the regular routine in 
classes for deaf children, the validation subjects used the Program 
in their classrooms under the supervision of their teachers. The 
teachers were asked to observe the chilfCren's reactions and comment 
w4p on these observations. 

(2.871) Individual Orientation. Ten demonstration frames , 
representing most of the frame types appearing in the Program, were 
used to teach the procedure. Directions were accomplished by a 
specified sequence of individual demonstration and guidance, with the 
gradual withdrawal of assistance. It was anticipated that, upon com- 
pletion of these frames, the procedure would be learned by most of 
the children. However, children were individually supervised; errors 
were recorded; and any assistance necessary was provided through 
Section C, Part 1 of Set One. It is at this point that the two tracks 
diverge; therefore, decisir.ns regarding track assignments were neces- 
sary at this time. 



(2. 872) Classroom Participation. Upon completion of individual 
orientation and track assignment, a member of the project staff super- 
vised the beginning use of the Program in each classroom and instructed 
the teacher in the procedure to be foilowed. In addition, a supervisor 
was available in each school to assist teachers with procedure, schedul- 
ing and materials for the duration of the validation period. 

In the classroom the children worked independently in their usual 
setting for individual activities. In most classes individual desks were 
used, but in a few rooms it was necessary to use large tables. Each 
day the teacher distributed Program books and reinforcing devices; 
checked to see that children were ready to work; and gave a starting 
signal. Each child worked at his own rate until the signal to stop was 
given- -usually five to ten minutes. He then marked his place and the 
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teacher collected materials, A record was kept of the approximate 
total work tiine for each child. 

When a child had completed one book of the Program, he re- 
ceived the next one in the sequence --progressing through his desig- 
nated schedule for the main Program and the supplementary Subject 
Noun Program, if needed. At designated intervals in the Program, 
progress tests, covering the immediately preceding content, were 
administered individually by the project supervisor. As previously 
noted, decisions regarding the child's track assignment for the next 
Set were made on the basis of these tests. The teacher was pro- 
vided with a schedule designating the sequence of books and the 
various testing points for each child. 

(2, 873) Problems Associated with Classroom Use, On a num- 
ber of occasions, two to four week lapses of time intervened between 
the completion of one book and the starting of the next one in the 
series. These lapses were due to uncontrollable delays in the print- 
ing of Program materials. The basic problems were (1) matching 
the appearance of conductive e id non-conductive ink patches and (2) 
the contamination of the non-conductive patches by the conductive 
material. The series of frustrating problems encountered and the 
ultimate solution .are discussed in detaii in Appendix F. Perhaps the 
most unfortunate result of these delays was the loss of classroom 
time necessary for validation of the final portions of the Program- - 
Set Four and Book VI of Set Three, 

(2, 874) Modifications in the Presentation, During the trial 
arid revision phase of the Program, repeated efforts were made to 
anticipate problems which might arise in the target situation, e 
realized that, in spite of these efforts, unpredicted problems might 
aTise when the Program was subjected to regular classroom use. 
Therefore, it was decided to sacrifice uniformity in Program pro- 
cedure, from Set to Set, in order to allow for improvements in the 
method of presentation. As teachers and project staff observed 
problems, possible soliitions were developed and tested in the class- 
room until a desirable solution was found. 

In the early portione of the Prog*^am, frames which required 
the student to write also appeared in a separate throw-away writing 
book. On designated frames in the main book, the student was cued 
to write on an identical page in his writing book. Many students had 
difficulty in shifting back and forth between the main book and the 
writing book at the appropriate places. Some continued in the writing 
book beyond the appropriate point, thus encountering writing frames 
out of sequence. The difficulties involved in distributing the matched 
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sets of materials - -plus marking and finding the place in both books -- 
consumed a disproportionate amount of classroom time. In addition, 
management of materials and schedules was difficult for the teachers. 

The project staff felt that confusion on the part of the .‘.tudents 
probably interfered with learning. In addition, children appeared to 
dislike the writing frames. In any case, the waste of classroom and 
tead er time resulting from the use of the dual-book arrangemeni , 
could not be tolerated. The ultimate solution involved the use of 
erasable plasticized writing pages within the main book, thus elimi- 
nating the need for separate writing books. Teachers were unanimous 
in their choice of the single book arrangement as opposed to the earlier 
edition, and children no longer appeared to dislike the writing pages 

(2, 88) Validation Testing . In order to evaluate learning result- 
ing from the use of the Program, pretest and posttest measures were 
obtained for each area of instruction. The three areas in which vali- 
dation testing was carried out w'ere verb vocabulary, sentence struc- 
ture and noun vocabulary. Of necessity the sentence structure pretest 
was administered after the students had completed Set One of the main 
Program and had met the requirement in subject nouns - -either by 
mastery on the pretest or by completing the supplementary Subject 
Noun Program. All tests were administered on an individual basis 
by trained personnel. Specification of the positions of the validation 
tests can be found in Appendix A. 

The 54-item Unit Verb Vocabulary Test, which served as the 
primary validation instrument, covered all four Sets of the Unit, At 
the time the pretest was administered, it was expected that all four 
Sets would be validated. However, because of delays encountered 
with the printing of Program materials. Set Four could not be used 
in the classrooms. Since all 54 items had been administered in pre- 
testing, the same procedure was followed in posttesting. Both pretests 
and posttests were then rescored as 40-item tests, omitting the fourteen 
items which related to Set Four, 
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RESULTS 



Data were collected primarily for the purpose of validating the 
Program Unit, Pretest and posttest measures were obtained in each 
of the three areas of instruction--verb vocabulary, noun vocabulary, 
and sentence structure--on each of the four sample groups. Additional 
data v/ere collected to permit examination of the relationship between 
characteristics of the sample and their performance on the Program. 



(3. 10) Program Effectiveness 

(3, 11) Summary of Pretest-Posttest Differences, The sign 
test (11) was used to make the twelve possible comparisons between 
pretest and posttost scores (three instruction areas times four sample 
groups). The hypothesis predicted that posttest scores would be higher 
than pretest sco es; therefore, the region of rejection was one-tailed. 
The number of positive signs (posttest higher than pretest) and their 
associated probabilities for each area are shown in Tabie III. 

It can be seen that, in each of the three schools and at both levels, 
the differences were significant beyond the . 05 level in all three areas 
of Instruction, Where the number Oa subjects is greater than 25, the 
probability levels are quite small, due in part, to a difference in the 
sign test formula for large samples. Of the total of 212 possible pre- 
test-posttest comparisons, 197 posttest scores were higher than the 
corresponding pretest scores, nine were ties, and six posttest scores 
were lower than the corresponding pretest scores. Since significant 
learning took place both in state residential schools and in a private 
day school; in schools located in different states; and with children who 
exhibited a wide range in chrciiological and mental ages, it would appear 
that these programmed materials have widespread applicability. 

(3. 12) Validation Criteria . The effectiveness of programmed 
instruction may be evaluated in several ways. One may establish an 
a priori final performance criterion, e. g. , 90% mastery, and ask what 
portion of the students met this criterion on the posttest, or one may 
examine the extent the gain from pretest to posttest resulting from 
the program. 

It should be noted that achievement of mastery and gain standards 
may be influenced by preknowledge of the program content as determined 
by pretest performance. If the level of preknowledge is low, it may be 



Table III. Sign Test Summary of Pretest-Posttest Cbanges for 

the Three Areas of Instruction for all Sample Groups, 
N = number of pairs excluding ties; x = number of 
positive signs. 



Level School 


N 


X 


P (one-tailed) 


Verb Vocabulary Test 








I I 


31 


30 


<. 00003 


1 II 


13 


12 


. 002 


I III 




5 


. 031 


II I 


29 


29 


<, 00003 



Subject Noun Teat 
I I 


29 


29 


<. 00003 


I II 


13 


13 


<. 001 


I III 


5 


5 


. 031 


II I 




9 


. 002 



Sentence Test 
I 


I 


28 


26 


<. 00003 


I 


II 


11 


9 


. 033 


I 


III 


5 


5 


. 031 


II 


I 


25 


25 


<. 001 



^Of the 29 children in Level II, 20 were exempt from the sup- 
plementary Subject Noun Program. 
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difficult to meet high final performance standards in a limited amount 
of time. However, a low pretest score is an advantage in meeting 
gain standards in that it provides a wide margin in which gain can 
occur. Conversely, a high pretest score may be an advantage in 
attaining high final performance criteria, but it restricts the margin 
in which gain can occur. Thus it would appear to be difficult to 
achieve both mastery and high gain. Nonetheless, attainment of both 
standards was attempted for the two instruction areas of the main 
Program, verb vocabularvr and sentence structure. Attainment of 
only one standard, high final performance, was attempted for the 
supplementary Program in subject nouns. The reasons for this de- 
cision will be explained in (3, 122) Nor ’''ocabulary. 

Two factors are involved in the selection of a final performance 
criterion for program effectiveness! the proportion of the pppulation 
expected to meet the mastery level and the designation of what con- 
stitutes masteiy. In an area such as acquisition of language by deaf 
children, it would appear necessary to set a goal which can be achieved 
by the majority of the children; to require that all children achieve the 
goal would be unrealistic. When the content of tl\e material being pre- 
sented will form <^he bases for further learning, mastery should be de- 
fined rather rigorously. The final performance criterion selected was 
80/90--80% of the validation subjects achieving a maste~y score of 90% 
On the Program pasttests. 

Two measures of gain were selected. One is a measur of pro- 
gram evaluation proposed by McGuigan (10) in which the actual gain 
(difference between posttest and pretest means) is compared with the 
possible gain (difference between the maximum score and the pretest 
mean). The resultant values range from 0 to 1. When the value of 
(Mg - Mj^ ) / (p - Mj) is . 5 or greater, the program is considered sat- 
isfactory. The second gain measure, proposed by Blake (1), is a modi- 
fication of the McGuigan gain ratio. In order to allow for the students' 
preknowledge, the term (Mg - Mj^ ) / P is added lo McGuigan's ratio 
with resultant values ranging from 0 to 2. Blake established a value of 
1.2 Or greater as satisfactory, 

(3, 121) Verb Vocabulary. Application of the three validation 
criteria to the maj or segment of the Pi ogram, verb vocabulary, is 
found in Table IV. 
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IV. 



Application of the Validation Criteria to the Verb 
Vocabulary Program for all Sample Groups, 
Perfect score = 40; chance = 10. 





N 


Median 

Post 


Maste ry 


Means 
Pre Post 


McG 


Gain 

. Blake 


Level I 
Scliool 
1 


31 


38. 2 


o8% 


10. 9 


34.4 


.81 


1.40 


II 


13 


35. 0 


46% 


10. 4 


32. 5 


. 75 


1. 30 


III 


5 


37. 8 


100% 


9.4 


37. 6 


. 92 


1. 63 


Level II 
School 
I 


29 


39. 4 


97% 


13. 6 


38. 7 


. 95 


1. 58 



As can be seen, there was a broad range in mastery achievement. 
Tv^o of the four sample groups exceeded the 80/90 criterion by a wioe 
margin; however, in one of the other sample groups only 46% of the 
students achieved mastery. Of the two groups which met the mastery 
criterion, one (School III) was a somewhat biased sample. The other 
group (School I, Level II) consisted of children who, as ft group, were 
older and had more school f*xperience than the other children. Exam- 
ination of the pretest. means makes it apparent that the differences in 
m.astery cannot be attributed to large difierences in initial performancu 
The Fisher exact prbbability test (11) was applied to the proportion of 
School I, Level I children achieving mastery (68%) and the proportion 
of Level II children achieving mastery (97%), The resultant p value 
of , 004 indicates that a significantly greater proportion of Level II 
children achieved mastery in verb vocabulary. 

^Because of problems with the printing of Program laterials, 
validation was begun later in the school year than originally pianned. 
School III was the last school to begin the Program and sufficient time 
was not available for all subjects to finish. Only the five children who 
worked rapidly were able to finish, judging from progress tests ac- 
companying each Set, three of the slower children would have achieved 
mastery and two would not. 
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The McGuigan gain ratios, which range from , 75 to , 95, exceed 
the , 5 criterion value for every sample group. Similarly, each of the 
modified gain ratios for the four groups exceed the critei-ion of 1 2 
established by Blake. These gain- in verb knowledge demonstrate the 
efficacy of programmed instruction as a tool in the acquisition of lan- 
guage by deaf children, 

In stirnmary, both gain standards v/ere met by all four sample 
groups, but the mastery standard was met by only two of the groups. 

It is important to note that the mean pretest scores, except for the 
13. 6 mean of the Level II children, were at the chance level of 10. 

Thus, the pretest and posttest verb scores illustrate the paradox 
that, while a low level of preknowledge may be advantageous in achiev- 
ing gain standards, it may be a disadvantage in achieving mastery 
standards . 

(3. 122) Noun Vocabulary. It was anticipated that many of the 
subjects selected for validation of the main Program would demon- 
strate sufficient preknowledge of the noun vocabulary to render them 
inappropriate for validation of the supplementary Subject Noun Program 
However, mastery of the noun vocabulary was considered necessary 
for success in the sentence structure portion of the main Program; 
therefore, only validation subjects who met the 90% mastery criterion 
on the 28-item Subject Noun pretest were exempted from this Program. 
Since the sole purpose of this Program was to insure initial behavior 
needed for the sentence structure portion of the main Program, the 
primary concern was the achievement of mastery by as many subjects 
as possible rather than the extent of the gain. In i able V the valida- 
tion data are displayed separately for the total group (Program partici- 
pants and students exempted) and for the group which received the 
Program, The exemptions, resulting from mastery scores on the 
pretest, consisted of one child at Level I and 20 children at Level II, 

The 80/90 criterion was met by thL-ee of the four sample groups 
from the total sample. For children who took the Program, the 
mastery criterion was met by the Level I groups in two schools and 
missed by 2% by the Level II group. The notablG exception to this goal 
is found in the School II data where the percentage of children achieving 
90% is unacceptably low. It should be noted that the pretest scores for 
this group were lower than any of the others. 

While only one student in Level I was exempted from the Subject 
Noun Program group, approximately two-tb.?rds of the students in 
Level II were exempted. It appears reasonable to assume that those 
who achieved mastery on the pretest were the better students and that 
their exclusion biased the Level II Program sample. Therefore it was 
anticipated that compariBon of Level I and Level II Program groups 
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from School I might fhow masterv and gain values higher at Level I. 
However, the data for the total groups show lievei II considerably 
higher in percentage of smdents achieving mastery. 

Although gain was not of direct concern, it is of intereat lu no' e 
that the value yielded by the application of McGuigan's ratio exceeded 
the . 5 criterion in every case. However, the effect of higher pretest 
scores upon Blake's modified gain ratio is evident--the resulting \ alue 
achieved the 1. 2 criterion only at Level I in School I. II .should be 
noted that the highest value poseiblc at School III was 1. 33--assuming 
all the children had achieved perfect scores. 

The primary goal--achieving the 80/90 criterion with the total 
sample --was reached with threu of the four groups. The children in 
School II scored lowest on all three performance criteria. This find- 
ing is not surprising since they held a similar rank on the verb vocab- 
ulary tests. In both instruction areas, the percentagve of children in 
this group attaining mastery was about half that desired. Obviously, 
the failure of the majority of the children in School TT to achieve 
mastery in verb and noun vocabulary would be expected to influence 
performance adversely on the subsequent sentence structure portion, 

(3. 123) Sentences, The evaluation of the sentence structure 
portion of the main Program is displayed in Table VI. 

The 80/90 criterion was achieved with t, *ee of the four groups, 
the exception being Level I in School II, As noted in the p'*evious 
section, this group did not achieve mastery in either of the component 
vocabulary areas--nouns or verbs. Therefore, it is not surprising 
that achievement on sentence structure was unsatisfactory. Hov/ever, 
the percentage achieving mastery (58%) was higher than for verbs (46%) 
or nouns (39%). No other group evidenced this relationship. 

All gain ratios, computed according to McGuigan, exceeded the 
, 5 criterion. However, the effect of the higher pretest scores on 
Blake's ratio is evident. Although three groups met the mastery cri- 
terion and all four met McGuigan's criterion, only School III met 
Blake's criterion cf 1.2. This group scored lowest on the pretest and 
all achieved perfe''t scores on the posttest. Even with a mean posttest 
Score of 19. 6 out of 20 items, Level II missed the 1.2 criterion by a 
small margin. 

Although high pretest scores were expected on the Subject Noun 
test, such was not the case with sentence structure since pretest 
scores during the trial and revision phase of the project had b€‘en 
close to chance. There are several possible explanations for these 
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Table VI. Application of the Validation Criteria to the Sentence 
Structure Program for all Sample Groups, Perfect 
score = EO; chance = 6.7, 





N 


Median 

Post 


Maste ry 


Means 
Pre Post 


McG 


Gain 

. Blake 


Level I 
School 
I 


31 


19, 6 


81% 


11.8 


17. 9 


. 74 


1. 05 


II 


1 2 - 


18. 0 


58% 


9. 1 


15. 9 


. 63 


. 97 


III 


5 


20i 0 


100% 


8. 6 


20. 0 


1. 00 


1. 57 


Level II 
School 
I 


29 


19. 8 


93% 


14. 3 


19. 6 


. 92 


1. 18 



*One child in School II was unavailable for the Sentence Posttest. 

higher scores which occurred during the validation phase: 

1, It is possible that the sample used in trial and revision was 
not representative of the laiger validation sample. 

2, Whei'eas chance performance in the two vocabulary areas 
was 25%, chance on the sentence structure test was 33 1/3%. 
Furthermore, the child who recognized only one of the two 
principal stimulus components increased his probability of 
attaining a correct answer without comprehension of the 
subject-verb relationship from 33 1/3% to 50%, 

3, Unlike the two vocabulary tests, it was possible for the 
student to learn the required task, understanding the sub- 
ject-verb relationship, from the test itself, 

(3, 124) Summary of Validation Criteria, Seventy-seven per- 
cent of the total sample of 78 children achieved mastery in Verb 
Vocabulary, Of the 57 children who took the Subject Noun Program 
74% achieved mastery. Thus, including the 21 children who met the 
mastery criterion on the pretest, 81% of the total sample achieved 
mastery of nouns before proceeding to the sentence structure portion 
of the Program, Eighty-three per cent of the total sample achieved 
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mastei ■ on sentence structure. Of the twelve instances in which the 
noastery criterion was applied (three instruction areas times four 
sample groups) the criterion was met in eight instances. Application 
of the gain ratio criteria to the two instruction areas of the main 
Program, verb vocabulary and sentence structure, disclosed that all 
four sample groups met McGuigan’s criterion in both areas of instruc- 
tion, When Biake's more rigorous criterion was applied, all four 
groups attained the standard in verb vocabulary, but 'nly one group 
in sentence structure. In brief, the gain ratio criteria were applied 
16 times. The validation sample groups met these criteria in 13 of 
the 16 applications. 

(3. 13) Relevance of the Program. It is of interest to icamine 
the relationship, if any, of the materials presented in the Program to 
those utilized in conventional methods. While the fmal design of the 
project was not intended to p/ovide for direct examination of thi ; re- 
lationship, data are available which appear to be pertinent to the 
question. The Unit Verb Vocabulary test scores from School I were 
utilized in this endeavor since (1) pretest measures were available 
from three levels for comparative purposes; and (2) the sample sizes 
of 31, 29, ai.d 26 from School I were large enough to allow logical in- 
ference, The median verb vocabulary scores for the three levels are 
presented in Table VII, 



Table VII, Median Pretest and Posttest Unit Verb Vocabulary 
Scores from School I. Perfect score = 40; 
chance = 10. 





Level I 


Level II 


Level III 


Pretest 

Median 


10.2 


12. 0 


29.0 


Posttest 


Median 


38.2 


39.4 


N. A. * 


N 


31 


29 


26 



*The Verb Program was not administered to the Level III 
children because of their relatively high pretest scores. 



Examination of the pretest medians from the Level I and Level II 
sample groups makes it evident that these children were not familiar 
with the verb vocabulary of the Program. In contrast, the Level III 
pretest median (72%) indicates a moderately high level of preknowledge. 
The fact that the children at Level III had leai d a considerable portion 
of the verb vocabulary appears to demonstrate tiie relevance of the Pro- 
gram content to academic achievement. 

The merit of the Program can also be evaluated by comparison of 
the posttest median of 39.4 for Level II with the pretest median of 29. 0 
fur Level III. '•* Since the Level II students scored 30% higher in May 
than the Level III students in December, it seems unlikely that the high 
achievement of the Level II students would have occu3 ed without the 
Program, A parallel can be found in the May posttest median of 38. 2 
for Level I and the December pretest median of 12.0 for Level II. It is 
even less likely that the gain exhibited by the Level I children would 
have occurred without the Program. Since the Program vocabulary 
was learned at levels where such verbs apparently are not usually ac- 
quired, we feel that this is important evidence that verb vocabulary 
can be learned earlier and at a more rapid rate through the medium 
of programmed instruction, 

(3. 14) Sets of the Main Program. The divisions of the main 
Program which were administered for the purpose of validation con- 
sisted of three Sets --each designed to teach four verbs at a high level 
of generalization and sentence structure. Validation of verb vocabulary 
was based on a comprehensive test which covered the vocabulary from 
all three Sets. In order to study the achievement on verb vocabulary 
for individual Sets, the items pertaining to each Set were treated as sub- 
tests of the overall Unit Test. A series of sign tests (11) were performed 
on the pretest and posttest s cores within each sub-test. The resultant 
data, which appear in Talle VIII, show that all four groups of children 
demonstrated significant learning on all three Sets, 



In order to study the relative achievement among the three Sets-- 
having different possible scores--mean scores were conver.„d to per- 
centages. None of the differences among the pretest scores for the 
three Sets were significant within any of the four subject groups, nor 
were the posttest differences significant within groups. The pooled data 
show pretest percentages correct of 34%, 29%, and 25% for SeLs One, 
Two, and Three respectively, and posttest percentages of 88%, 90%, and 
91%, It is interesting to note that the downward trond--from Set One 
through Set Three --within the pretest percentages is reversed for the 
posttests , Thus the pretest-posttest differences within each Set (547o, 
61%, and 66%) show the greatest gain in Set Three, i,e, , an apparent 
trend toward iiicreased learning as the children progressed through the 
program. 
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Table VIII. Sign Test Summary of Pretest-Posttest Changes 
for the Sets of the Verb Vocabulary Program 
for all Sample Groups, N = number of pairs 
excluding ties; x = number of positive signs. 



Level 


School 


N 


X 


P (one-tailed) 


Set One 










I 


I 


30 


30 


<, 00003 


I 


II 


13 


13 


<. 002 


1 


III 


5 


5 


. 031 


II 


I 


27 


27 


<. 0003 



Set Two 
I 


I 


2 0 


27 


<.00003 


I 


II 


12 


11 


. 0()3 


I 


III 


5 


5 


. 031 


II 


I 


29 


28 


<.00003 



Set Three 
I 


I 


29 


27 


<.00003 


I 


II 


13 


12 


. 002 


I 


III 


5 


5 


. 031 


II 


I 


29 


27 


<.00003 



(3. 15) Children Excluded from the Sample. It will be recalled 
that children who do not demonstrate word recognition ability at a 
minimal level, regardless of the reasons, are not considered a part 
of the target pupulation for the Program; therefore, those children 
in the participating schools who failed the Word Recognition Test were 
not included in the va'idation sample. It had been predicted that such 
children would not succeed in the Program; however, they were included 
in all validation activities in order to test this assumption. 
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The 22 children from the three schools who did not meet the 
word recognition requirement for admissior tO the validation sample 
performed as follows: 

Eight children did not appear to comprehend the procedure and 
scored at a chance level on the first Progress test (covering 
only Set One vocabulary). Participation in the Program was 
discontinued for these children at the end of Set One, 

Seven children did not progress beyond Set One in the Program 
although they continued to participate until tiie end of the school 
year. Two of these children scored 62% correct on the first 
Progress test and the other five scored less than 50%, 

Four children achieved satisfactory scores on the first Progress 
test (88% - 94%) but worked so slowly that they did not progress 
beyond Set One. They became increasingly disinterested and 
were working more slowly at the close of the validation period 
than at the beginn'ng. 

Three children were able to complete the Program and the vali- 
dation posttests. The highest verb vocabularv posttest score 
achieved by any of these children was 55% correct. 

In summary, although children \vho did not demonstrate pre- 
requisite word recognition ability varied in their performance, none 
of them performed satisfactorily. Therefore, the assumption that 
children who fail the Word Recognition Test should not be considered 
candidates for the Program was verified. 



(3, 20) Program Efficiency 

(3,21) Time Re qulremeri.ts for the Program. In addition to the 
effectiveness of a program, one must consider its efficiency- -the 
amount of time required to complete the program. In order to pro- 
videi for evaluation of this aspect of the project Program, classroom 
teachers recorded the time spent by the students working in the Pro- 
gram books. The resultant data for the main Program, verb vocabu- 
lary and sentence structure appear in Table IX, 



Table IX. 



Program Time in Minutes for the Verb Vocabulary- 
Sentence Structure Program for all Sample Groups. 





N 


Range 


Mean 


All Subjects 


78 


129-495 


281 


Level I 
School 
I 


31 


183-495 


342 


II 


13 


224-329 


276 


m 


5 


191-308 


267 


Level II 
School 
I 


29 


129-335 


224 



It can be seen that the average time reqxiired to complete the 
main Program was less than five hours --the equivalent of approxi- 
mately one school day’s instructional time for most of the children 
involved. The slowest child spent less than two days. As might be 
expected, the Level I children spent longer on the average and had a 
wider range than the Level II children. 

Data were also gathered on the amount of time required by the 
58 children who needed the supplementary Subject Noun Program. 
Forty-seven needed the entire sequence; their average time was 63 
minutes. The eleven children who needed only a portion of the sup= 
plementary Program averaged 28 minutes. Thus, for all subjects, 
the amount of work time spent in the main Progreim (verb vocabulary 
and sentence structure) plus the supplementary Program (nouns) was 
the equivalent of two school days or less since the time required by 
the slowest child was ten hours. 

Additional time was required for daily distribution and col- 
lection of Program materials, IBT-TS, validation testing and peri- 
odic progress testing. Although time spent on these activities was 
not recorded, it was estimated on the basis of observation to have 
averaged about three and one -half hours per child. Therefore, the 
total time required for the administration of the Program and its 
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related activities was the equivalent of approximately two school days 
for the majority of the children — for the slowest child less than three 
days. 



Inforndation gained from 'observation, classroom teachers, and 
time data indicate that the total time requirements for the future ad- 
ministration of the Program could be substantially reduced, especially 
for Set One and Subject Nouns. Because of the children’s confusion 
with the procedure involved in the writing task in Set One and Subject 
Nouns, the Program presentation was modified so as to simplify the 
procedure in Set Two and Set Three. Many of the teachers reported 
that their children handled the modified materials much more ef- 
ficiently. Time data for the three Sets of the main Program support 
this observation. While there was a decrease of only 17% in the num- 
ber of Program frames from Set One to Set Two, the decrease in 
Program time for Level I at School I was 48%. The differences be- 
tween Set Two and Set Three were very small- -a 12% reduction in the 
number of frzimes and 18% in time. Similar trends were observed in 
the other subject groups. It is logical to assume that the increased 
efficiency beginning with Set Two is dxie in part to practice effect. 
However, it is also reasonable to assxime that the simplification of 
procedure exerted a favorable influence upon efficiency- -especially 
since the teachers reported a marked improvement beginning with 
the use of the modified materials. 

With the original presentation, used in Set One, there was also 
confusion involved with the daily distribution and collection of ma- 
terials. Teachers reported a substantial reduction in the time re- 
quired for this aspect of Program use with the introduction of the 
modified presentation in Set Two« In our judgment the total time re- 
quired for the administration of Set One and Subject Nouns could be 
reduced by about 50 minutes with the use of the modified presentation. 

Other smaller reductions in the total time required for the Pro- 
gram and its related activities could be achieved in two areas. In 
regular classroom use the Progress pretests, immediately preceding 
each Set, could be omitted. It is not recommended that the Progress 
posttests be omitted as these provide the teacher and the child with 
information regarding progress at various points in the Program. In 
the future, the noun division of the IBT-TS, which proved to be in- 
effective, could also be omitted. These two revisions in the total 
Program schedule should reduce the time requirements by approxi- 
mately 30 minutes. 

In summary, it is estimated that the Program, as it was field 
tested, required an average total administration time of approximately 
seven hours for the fastest group (School I, Level IE) and about eleven 






hours for the slowest group (School I, Level I), The slowest individual 
reqxiired approximately 14 hours. In view of the achievement dis- 
cussed in the previous sections, which took the participants in Levels I 
j and II beyond the average achievement of the Level III children in the 

same school, the Program would appear to be an efficient utilization 
of classroom time. In our judgment, the recommended modifications 
in the Program presentation and in various testing activities could re- 
duce the average time requirement by approximately one and one -half 
hours for a group of children similar to the slowest group, 

(3,22) Program Time and Achievement, In order to study the 
i relationship between Program time and achievement, the combined 

' scores for the two areas of the main Program, verb vocabizlary and / 

sentence structure, were divided by quartiles and the respective timje 
I data were compared. While it was felt desirable to use data from th*e 

I total sample, these data were not amenable to grouping by achieve - 

[ ment quartiles. For example, there were no children below mastery 

in School HE; therefore high and low achievement groups would have 
[ been drawn predominately from different schools, Sinoilar problems 

I were encountered with grouping by the time data. Therefore, we 

I elected to consider only the d:ita from School I and to examine the data 

j for the two levels separately. The time required on the main Program 

I for the upper and lower 25% achievers, according to their combined 

\ scores for verb vocabiilary and sentence structure posttests, appear 

I in Table X, 

% 

\ 

t 

i 

\ 

I 

f Table X, Program Achievement and Time for the Upper 25% 

I and the Lower 25% Achievers in Levels I and II at 

I School I, Perfect score (verb vocabulary - 

j sentence structure) = 60; chance = 17, 





N 


Posttest 

Mean 


Time 

Mean 


in Minutes 
Range 


Level I 










Upper 25% 


8 


59.9 


324 


183-495 


Lower 25% 


8 


35,0 


349 


218-492 


Level n 










Upper 25% 


7 


60,0 


226 


146-303 


Lower 25% 


7 


54,7 


259 


142-335 
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At Level I the children in the upper achievement quarter took 
only 25 minutes less time to complete the Program than the children 
in the lower quarter. However, the achievement differences are sub- 
stantial, All the children in the upper quarter achieved mastery; 
none of those in the lower quarter achieved mastery. At Level II 
children in the upper quarter took 33 minutes less than those in the 
lower quarter. In contrast to Level I, only one child in the lower 
quarter failed to achieve mastery. 



While* within each level the children in the upper quarter tended 
to spend less time than those in the lower quarter, there is a much 
greater difference between levels — both mean times at Level I are 
longer than either of the means at Level II by approximately a 3:2 
ratio. As was noted above, children in the lower quarter at Level I 
are in an achievement class which is markedly different from the 
other three groups. Such factors as age and academic level may have 
had significant influence on their performance. These and other factors 
will be examined in (3,40) Characteristics of the Subjects, 

It is important to note that at Level I, both the fastest child and 
the slowest one were members of the high achievement group. Thus, 
while it is evident that as a group the highest achievers took less time 
to complete the Program than the lowest achievers, the data also 
demonstrate that high achievement is not based on a rapid work rate. 

In view of these findings, it is recommended that children not be 
pressured to work rapidly through the Program, but rather that they 
be encouraged to work carefully and accurately. 



(3,23) Standard and Accelerated Tracks, In order to permit 
greater efficiency and to avoid boredom for the more capable students 
and, at the same time, provide the repetition necessary for others, 
two tracks were designed for the main Program (2, 86, Track Assign- 
ment), Children in the Standard Track repeated certain segments of 
the verb vocabulary sequence while children in the Accelerated Track 
continued through the Program without repeating any material. 

Level n at School I was the only group which contained a sufficient 
number of cases in each track to permit comparison of the time re- 
quirements for the two different tracks. The time data and the average 
achievement on the main Program for the Level II children in the 
Accelerated Track, those in the Standard Track, and those who were 
shifted from Standard to Accelerated Track after Set One appear in 
Table XI, 
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Table XI. Program Achievement and Time for Level II 

Children in Standard Track, Mixed Standard and 
Accelerated Tracks, and Accelerated Track. 
Perfect score (verb vocabulary - sentence 
structure) = 60; chcince = 17. 



Track 


N 


Posttest 

Mean 


Time 

Mean 


in Minutes 
Range 


Accelerated 


16 


59. 1 


186 


129-245 


Mixed 


5 


59. 0 


252 


235-266 


Standard 


8 


56. 1 


289 


248-335 



It is readily apparent that utilization of the Accelerated Track 
by the more capable children at Level II did not adversely affect their 
performance and apparently resulted in considerable saving in time. 
The one non-mastery achiever at Level n was assigned to the Standard 
Track throughout the Program. At Level I 16 of the 49 children from 
all three schools were assigned to the Accelerated Track for some 
portion of the Program. Twelve of these achieved mastery of 54 or 
more. The scores of the four children who failed to achieve mastery 
ranged from 48 - 52. • 



(3.30) Initial Behavior Testing -Training Sequence 

Two initial behaviors (abilities prerequisite for success in the 
Program) were specified: (1) word recognition s kill at a miTiiTr>a1 
level and (2) the ability to associate pictures with reality. No attempt 
was made to overcome deficiencies in word recognition skill; children 
who did not demonstrate a minimal level of proficiency were excluded 
from the validation sample, although they participated in all Program 
activities (3. 15, Children Excluded from the Validation Sample). How- 
ever, children who demonstrated a deficiency in the ability to associate 
pictures with reality were not excluded. The Initial Behavior Testing- 
Training -Sequence was designed not only to assess ability in this area 
but also to provide instruction for children who demonstrated a deficiency. 



The IBT-TS consisted of two divisions. The first required the 
association of pictured objects (nouns) to real objects and the second 
the association of pictured actions (verbs) with live actions. Test 
scores were obtained by recording the child's first response to each 
item. Instruction was accomplished by providing feedback for each 
response and by assistance with items failed. 

Performance levels on the noun division were high. Fifty-seven 
children scored at the proficiency level (one or no errors) during the 
first session (Day I), Of the 21 children who made two or more errors 
during the first session, only one made ap many as two errors during 
the second session (Day II). No norni instruction was planned for Day III. 

In order to adapt the verb division of. the IBT-TS to varying levels 
of ability, three parallel sets of materials, each containing A, B, and 
C, Sections, were developed and administered on three successive days 
prior to beg inni ng the Program. Children who perform.ed satisfactorily 
on Day I or on Day II were exempted from Section A and Section B of 
the verb sequence on the following day. All students received Section C 
each day. 

(3.31) Learning from the Verb Division of IBT-TS. Of the total 
sample, a group of 23 children achieved proficiency (two errors or less) 
on Day I and were exempt from Sections A and B on the following days. 

A second group of 21 children achieved proficiency on Day II and were 
exempt from Sections A and B on Day III. A tMrd group of 34 children 
required the entire sequence. No School in children are included in the 
third group, since all of them achieved proficiency either on Day I or on 
Day n. The data for these three groups appear in Table XU. 

Within Group II the performance of Level I and Level II was very 
similar. Both levels evidenced a decrease in errors of more than 50% 
from Day I to Day II. The Day II errors for this group were similar to 
the Day I errors for Group I. Within Group m. Level II evidenced a 
consistent decrease in errors from Day I to Day II to Day HE. Although 
Level I showed an overall decrease in errors from Day I to Day III, 
there was a slight increase from Day I to Day II. The performance of 
the two levels was very similar on Day m. Only 16 of the 34 children 
in Group HE achieved proficiency on Day m. 

Of the total of 55 children who demonstrated a deficiency on Day I, 
37 achieved proficiency by the conclusion of the testing-trairdng sequence. 
Including those whose performance was acceptable on Day I, 60 of the 78 
children demonstrated ability to associate pictures with experience be- 
fore beginning the Program^ and 18 children (all from Group HE) still 
had not attained proficiency when they began the Program. 



Table XII. IBT-TS Errors on the Verb Division, Sections A and 
B, for all Subjects. Total N for Level I = 49; 

Level n = 29. 









Errors 








N 


Day I 


Day n 


Day in 


Proficiency 


Group I 
Level I 


.9 


7% 






100% 


Level n 


14 


7% 






100% 


Group n 
Level I 


15 


; 25% 


9% 




100% 


Level n 


6 


24% 


9% 




100% 


Group in 
Level I 


25 


26% 


29% 


17% 


44% 


Level n 


9 


33% 


■23% 


16% 


56% 



(3. 32) Influence of IBT-TS on Program Achievement. Of the 60 
children who, at some point, achieved proficiency on IBT-TS, 80% 
achieved mastery in the main Program. Of the 18 children who did not 
achieve proficiency on IBT-TS, 64% achieved mastery in the Program. 
This observation can be contrasted with the finding that none of the 
children who failed the Word Recognition Test achieved mastery on the 
Program. Apparently, of the two behaviors predicted to be prerequisite 
for Program success, only word recognition skill was in fact essential. 
Proficiency on IBT-TS, although beneficial, did not prove to be neces- 
sary. 



(3.40) Characteristics of the Subjects 

Tn an attempt to determine the target population appropriate- for 
the Program, varioxis characteristics of the validation sample were 
studied in relation to Program performance. The following variables 
were examined: chronological age, academic level, psychometric level, 
race, sex, and hearing level. Chronological ages were recorded as of 
January 1, approxinaately the time when most of the c hil dren began the 
Program. Grade placement, which had been assigned by the individual 



schools, determined academic level. It is likely that criteria for level 
placement varied from school to school. In order to obtain uniform 
measures of learning ability across schools, three psychometric test 
items were administered: (1) the Seguin Form Board from the Grace 
Arthur Point Scale of Performance, Revised Form IE, (2) Picture Idenr 
tification and (3) Picture Association from the Kiskey-Nebraska Test of 
Learning Aptitude (2, 84, Subject Information). Subjects were selected 
without regard to race or sex. Hearing level was estimated by averag- 
ing ISO response levels at 250 and 500 A portion of the data on the 

above characteristics for each of the four sample groups appears in 
Table XHI. 



The Program sample included children at two academic levels 
with a wide range in chronological age --from age five at Level I to age 
ten at Level II. The range in level for each of the three psychometric 
items was even wider. Of the twelve possible comparisons between 
chronological age and psychometric level (three psychometric items 
for each of the four groups) the mean psychometric level was lower than 
the corresponding chronological age in ten instances. In all instances 
the lower limit of the psychometric range was below that of the corres- 
ponding chronological age, and in all but one instance the psychometric 
range was wider. 

In order to study the effects of the various subject characteristics 
on achievement, the data were divided by quartiles. As in the previous 
section, (3.22) Program Time and Achievement, the data relating 
achievement to other variables were not amenable to quarter grouping 
when the total sample was considered. Therefore, School I with its 
large sample again was selected for examination. This procedure had 
the additional benefit of allowing comparisons with previously established 
quarter groups. 

(3.41) Chronological Age and Academic Level. The effects of 
aga academic level on achievement were examined by means of 
quarters selected according to age within each of the two academic levels 
at School I. The ages and corresponding achievement data appear in 
Table XTV. 
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Picture Association) in Years and Months for all Sample Groups. 



Age Form Board Identification Association 

Mean Range Mean Range Mean Range Mean Range 
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Table XIV. Program Achievement for Age Groups Defined by 
Quartiles, Level I and Level II, School I. Perfect 
score (verb vocabulary - sentence structure) = 60; 
chance = 17. 





Age 


Achievement 


N 


Mean Range 


Mean Range Mastery 



Level I 


8 


5-2 


5- 0— 


5- 7 


47.2 


17-60 


50% 


7 


6-0 


5- 9— 


6- 2 


53.0 


29-60 


71% 


8 


6-6 


6- 3— 


6- 9 


55.0 


24-60 


88% 


8 


7-8 


6-11 — 


9- 0 


53.0 


41-60 


62% 


Level n 


8 


6-5 


6- 1— 


6- 8 


56.9 


43-60 


88% 


7 


7-2 


6-10— 


7- 8 


58.4 


55-60 


100% 


7 


8-3 


7- 8— 


8-10 


58.0 


56-59 


100% 


7 


9-4 


8-11— 


10- 0 


59.4 


57-60 


100% 



Within Level I, as age increases there is a corresponding increase 
in achievement through the first three quarters. However, with the 
foiirth group, which shows the largest increase in age, this trend is re- 
versed. The average achievement for the oldest group is the same as 
that of the second group, but the percentage attaining mastery is lower. 

It should be noted that all four age groups show wide range in achieve- 
ment- -all conta ining both non-mastery students and those with perfect 
scores. The range for the yoxmgest group extends from chance to per- 
fect performance. There is however, an overall trend toward a narrower 
range in achievement as age increases. 

Within Level II the data show a slight overall increase in achieve- 
ment from the youngest to the oldest group, with the two middle groups 
about the same. Since there was only one non-mastery student at 
Level n, the average achievement for each quarter is above mastery (54) 
and the corresponding mastery percentages all exceed the 80/90 criterion. 
The effect of age on achievement appears to be less evident at Level H 
than at Level I. The fact that age per se is of relatively small influence 
is demonstrated by the fact that the yoxmgest child (5-0) and the oldest 
child (10-0) both' achieved mastery on the main Program. 
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The effect of academic level on achievement can be seen by exam- 
ination of comparable age groups from the two different levels. The 
third quarter of Level I, when compared to the first quarter of Level II, 
showed slightly lower achievement, although the mastery percentage 
was the same. When the fourth quarter of Level I is compared with the 
second or third quarter of Level II, it is also apparent that these Level I 
subjects scored lower in achievement and the percentage of mastery was 
smaller. Thus children at Level II, when compared with their age peers 
at Level I, performed somewhat better. However, the greatest differ- 
ence in achievement between the two Levels, age constant, is not as 
large as the effect on ^achievement associated with age differences with- 
in Level I. 

It appears that evidence of an interaction between age and academic 
level can be found in the fourth quarters --the oldest children at Level I 
and at Level II. It is reasonable to assume that some of the oldest 
children in these two groups are present in their respective grade levels 
because of poor classroom achievement. If this premise is correct, it 
is not surprising that the oldest children at Level I achieved a lower 
average score than younger children in the third quarter at the same 
level. However, every child in the oldest group at Level H, all of whom 
are above the appropriate age range for their level, scored above mas- 
tery. One can only speciilate as to the possible reasons, e. g. , length 
of classroom experience, for the high achievement of these cii-ildren. 

(3.42) Psychometric Level. Three psychometric test items for 
the Level I, School I children were examined in detail to explore possible 
relationships between learning ability and achievement. The Level II 
data were not similarly examined because all but one child achieved 
mastery. The data, which were grouped by achievement quarters, 
appear in Table XV. 

Increases in achievement scores appear to be unrelated to Form 
Board scores. While Picture Identification scores evidenced ian irregu- 
lar, but apparently positive, relationship with achievement, only the 
Picture Association mea.ns show a consistent trend. Eighty per cent 
of the children with Picture Association scores of 6-6 or better achieved 
mastery. However, it is important to note that the highest achievement 
quarter contained the lowest Picture Association score and the lowest 
achievement quarter, with no mastery achievers, contained a Picture 
Association score of 7-0. 



Table XV. Psychometric Levels (Form Board, Picture Identi- 
fication, Picture Association) of Achievement Groups 
Defined by Quartiles, Level I, School I. Perfect 
score (verb vocabulary - sentence structure) = 

60; chance = 17. 



N 


Achievement 


FB 

Mean 


PI 

Mean 




PA 


Mean 


Mastery 


Mean 


Range 


8 


35.0 


0% 


6- 3 


5- 3 


5- 2 


1 

0 

1 
1 

1 

o 


7 


54.9 


71% 


5- 4 


5-11 


5-11 


4_6--7-6 


8 


58.8 


100% 


6- 2 


5- 4 


6- 2 


4_6— 7-6 


8 


59.9 


100% 


5-11 


6- 2 


6- 5 


3„6— 8-6 



Since the achievement scores evidenced a greater relationship to 
Picture Association than to either of the other psychometric measures, 
we elected to regroup the data in quarters based on Picture Association 
scores. These data are presented in Table XVI. 



Table XVI. Program Achievement of Groups Defined by Picture 
Association Quartiles, Level I, School I. Perfect 
score (verb vocabulary - sentence structure) = 

60; chance = 17. 



PA Achievement 



N 


Mean 


Range 


Mean 


Range 


Mastery 


8 


4-4 


3_6--4-6 


44. 1 


17-60 


50% 


7 


5-5 


4-6--5-6 


52.0 


29-60 


62% 


8 


6-6 


6-0— 7-0 


55.7 


45-60 


71% 



As expected the data show an increase in achievement as Picture 
Association scores increase. Again it is evident that this relationship 
is weak; the lowest Picture Association quarter included both the lowest 
achiever and a perfect scorer. In the highest quarter, where 88% of the 
children achieved mastery, there was one failure. In comparison, only 
50% of the children in the upper quarter by Form Board scores achieved 
mastery. 

(3.43) Race, Sex and Hearing Loss. In order to examine the 
effects of race, sex, and hearing loss on Program performance, a 
series of matched groups were formed. Eleven Negro children were 
matched on the bases pf age, level and school with eleven White children. 
The achievement for the Negro students was 51. 1 in 300 minutes and for 
White students 52.4 in 264 minutes. 

The effects of sex and hearing loss on achievement were examined 
in a similar manner at Level I and at Level II in School I only. In regard 
to sex, the achievement for ten boys at Level I was 51.2 in 382 minutes 
and for ten girls was 49. 9 in 382 minutes. At Level II the achievement 
for 13 boys was 57.4 in 222 minutes and for 13 girls was 59. 3 in 227 
minutes. With respect to hearing loss, the achievement for the eight 
Level I children with the least loss (67db average) was 50. 6 in 350 min- 
utes; for the eight with the most loss (95db average) was 48. 0 in 354 
minutes. At Level II the eight children with the least loss (56db average) 
achieved an average score of 57.8 in 223 minutes as opposed to 57.0 in 
226 minutes for the eight children with the most loss (91db average). It 
is apparent that the differences in achievement and time attributable to 
race, sex, and hearing loss were small. With the exception of the dif- 
ference of 2. 6 between hearing loss groups at Level I, all differences 
in achievement were less than two points. 

(3.44) Summary of Subject Characteristics. In this and previous 
sections the influence of a numiber of variables upon achievement on the 
Program has been examinevi. The only efficient instrument for pre- 
dicting performance proved to be the Word Recognition Test. It will be 
recalled that, of the 22 children excluded from the validation sample on 
the basis of this test, none achieved mastery in the main Program. Of 
the 78 children who passed the Word Recognition Test, 76% achieved 
mastery in the main Program. 

Such variables as race, sex, amoTint of hearing loss, Seguin Form 
Board score, euid IBT-TS errors evidenced very little relationship to 
achievement. Other variables such as school, level, age and the.Hiskey 
Picture Association level have shown effects on achievement as large as 
five to ten points. However, there was always a wide range of scores 
within high and low achievement groups and frequently complete over- 
lapping of score ranges. Perhaps the fact that these characteristics 
did not constitute significant limitations within the sample is illustrated 



by S. W. in Level I at School I, the youngest child in the entire sample 
(5-0), with an 83db hearing loss and the lowest Picture Association 

j^ 0 coxded (3—6), who achieved a perfect posttest score of 60 for 

the main Program. 




CONCLUSIONS AND RECOMMENDATIONS 



(4, 10) Summary and Conclusions 

A unit of programmed instruction utilizing a simple, inexpensive 
feedback device was developed for young deaf children. The main Pro- 
gram consisted of two^ areas of instruction- -verb vocabulary at a high 
level of generalization and sentence structure, A third area, noun 
vocabulary, was provided in a supplementary Program for students 
who needed instruction in the noun vocabulary prerequisite to the sen- 
tence portions of the main Program, Terminal behavior was specified 
by means of three validation tests --one for each area of instruction. 

After extensive trial and revision of the Program materials, the 
final form was field tested with 78 children in the preparatory levels of 
two state residential schools and one private day school in the south- 
eastern area. Level I at each of the three schools and Level IE in one 
of the state schools comprised the four sample groups. Within these 
groups a satisfactory score on a simple test of minimiim word recogni- 
tion ability was the only criterion for admission to the sample. The 
subjects ranged from five years of age at Level I to ten years at Level IE, 
and from three and one -half to ten and one -half years on psychometric 
scores. 

The effectiveness of the Program was evaluated by validation pre- 
test and posttest measures for each of the three areas of instruction. 
Three validation criteria--one final performance criterion and two 
measures of gain — were applied to the test resxalts for each of the four 
sample groups. The final performance (mastery) criterion selected 
was 80/90--80% of the validation subjects achieving a mastery score 
of 90% on the Program posttests. The first gain ratio, proposed by 
McGuigan (10) compares the actual gain with the possible gain. The 
resultant values range from 0 to 1 with ,5 or greater considered satis- 
factory, Tfie second measure of gain, a modification of McGuigan's 
ratio proposed by Blake (1), is a more rigorous standard which pen- 
alizes the programmer for the students* preknowledge of the program 
content. The values range from 0 to 2 with 1,2 considered satisfactory. 
All three criteria were applied to the two instruction areas of the main 
Program for each of the four sample groups; however, the mastery 
criterion was the only goal for the sui>plementary Program. 



The mastery criterion was met by two of the four sample groups 
in verb vocabulary, by three groups in noun vocabulary, and by three 
in sentence structure. Mastery within sample groups ranged from 
46% to 100%. One sample group failed to meet the 80/90 criterion in 
any of the three areas of instruction. 

The McGuigan gain ratio criterion was exceeded by all four sample 
groups on both the verb vocabxilary and the sentence structure portions 
of the main Program. The same sample group which failed to achieve 
mastery on the verb, noun and sentence areas consistently evidenced 
less gain than the other three groups. 

Application of Blake*s ratio showed that all four sample groups 
exceeded the criterion in the area of verb vocabialary. However, only 
one sample group met the criterion in the area of sentence structure. 
Again, the sample group which failed to meet the mastery criterion and 
which evidenced the lowest McGuigan gain ratios, evidenced the smallest 
gain by Blake's ratio. 

The criteria were applied to the data from the Level I sample 
groups 21 times. In 15 of 21 instances, the criteria were met. The 
same criteria were applied to the Level II data seven times; six times 
the criteria were met. In the one exception, the criterion was missed 
by a very narrow margin. In summary, the validation criteria were 
applied in 28 instances to the data from the four sample groups. The 
Program met the standards in 21 instances. Of the total sample of 78 
children, 77% achieved mastery in verb vocabulary. 81% in nouns, and 
83% in sentences. Application of the sign test to the pretest and posttest 
scores showed significant learning by each of the four sample groups in 
all areas, of instruction. 

For the main Program, the estimated average time spent by the 
49 children at Level I was approximately 317 minutes; the 29 Level II 
children spent approximately 224 minutes. For the 47 children who took 
the entire supplementary noun Program, the estimated average time was 
63 minutes. The eleven children who needed only a portion of this Pro- 
gram averaged approximately 28 minutes. Additional time required for 
related activities, such as distribution of materials in the classroom and 
testing, was estimated at 210 minutes per child. It is estimated that the 
time required for the administration of the Program and its related 
activities averaged approximately seven hours for the fastest group and 
eleven hours for the slowest group. The slowest individual spent approx- 
imately 14 hours. 

The relationship between achievement on the Program and a number 
of variables was examined. Race, sex, amount of hearing loss, Seguin 
Form Board score, and RLcture Identification score evidenced very little 
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effect on achievement. School, age and Picture Association score 
appeared to have a larger effect on achievement. None of the variables 
examined evidenced sufficient relationship with achievement to be useful 
for predictive purposes. The only efficient instrument for predicting 
Program performance proved to be the Word Recognition Test. Of the 
22 children who failed this test, none achieved mastery. 

Since in 21 of 28 instances the criteria for effectiveness were met 
by the subject groups in the equivalent of approximately two school days, 
it appears reasonable to conclude that the Program teaching sequence, 
together with the modified method of presentation, constitutes an effective 
and efficient teaching technique. Inasmuch as the sample contained chil- 
dren from three schools, and high achievers were as young as 5. 0 years 
with psychometric age equivalents as low as 3.5, it also appears reason- 
able to conclude that the Program has widespread applicability for young 
deaf children. One may also conclude that the goal of developing pro- 
grammed instruction utilizing a simple, inexpensive medium ready for 
use in existing classrooms was attained. 

Two findings warrant restatement because of the impliciations for 
future application of this or similarly designed programmed instruction. 
First, the fact that the Level m children demonstrated 72% preknowledge 
of the Program content in verb vocabulary can be construed as evidence 
that the Program is relevcint to academic achievement. Secondly, the 
high achievement (96%) by the Level I children from the same school 
demonstrates that verb vocabulary can be learned earlier and at a more 
rapid rate through the medium of programmed instruction. The implica- 
tions of these findings constitute, in our judgment, a justification for 
further research and development of similar prograunmed Instruction 
units. 



(4«20) Recommendations 

During classroom use of the Program two serious problems arose 
which could be corrected in the future. First, because of delays in ob- 
taining printed materials, student use of the Program was interrupted at 
various points for considerable periods of time. Second, the procedure 
utilized for the writing frames in Set One and in the supplementary Pro- 
gram resulted in confusion for many of the children. A solution to this 
problem was found and materials for Sets Two and Three were modified 
to accommodate the new procedure. The effect of these two conditions 
upon Program achievement cannot be estimated, but it is reasonable to 
assume some adverse effect. 



Since the Program met the criteria for effectk-eness in 21 of 28 
instances, the possibility that it would have met all criteria with the 
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above conditions corrected shoiald not be ignoredo Theretore, we recom- 
mend that the Program be field tested again with a similar group of 
children under the following conditions: 

1. That the writing frames in Set One and the supplementary 
Subject Noun Program be changed to the format used suc- 
cessfully in Sets Two and Three; 

2. That all materials be obtained before caildren begin the 
Program; 

3. That Set Fo^ur, which could not be used during the first field 
testing, be included. 

If, in the second field testing, the Program does not meet the 80/90 
mastery criterion and Blake*s gain criterion, we recommend further 
trial revision. When the Program Unit has met both mastery and 
gain criteria, we offer the following recommendations: 

1. That the Program Unit be disseminated to school personnel 
who wish to use it; 

2. That the Program Unit be used as a base for programming 
more difficult langTiage concepts — those which require an 
understanding of simple intr a -sentence relationships (taught 
in the Program Unit) for their development, e, g. , verb 
tense, negation, and such words as have , is and can. 
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Appendix A- 1: Outline of the Prc^ram Unit 

UNIT VERB VOCABULARY PRETEST (1 6 Verbs) 

Set One (4 Verbs) 

Pretest 

Program: jump, eat, paint, wash 
Posttest 



Subject Nouns (11 Nouns) 

Pretest* 

Program: people, animals 
Posttest 



Sentence Struct\ire 
Pretest 

Program: subject-verb relationship 
Posttest 

Set Two (4 Verbs) 

Pretest 

Program: run, si 
Posttest 

Review: Sets One and Two 
Voca1>ulary 
Sentence Structure 

Set Three (4 Verbs) 

P^retest 

Program: : drink, sleep, write, cook 
Posttest 

Review: Sets One, Two and Three 
Vocabulary 
Sentence Structure 

Object Notins 
Pretest** 

Object noun j^t ruction 
Posttest 

Sentence Structure: Verb-object relationship 



*Chiidren who achicye inastery on this test oinit liie Subject Noun Program* 

**Children who achieve maatery on Ais teirt omit object noun instruction. 




Set Four (4 Verbs) 

Pretest 

Program: draw, cut, push, carry 
Posttest , \ 

Review: Sets One, Twpv^^^^ T 
■ Vocabulary 
Sentence Structure 
Subject -verb 
Verb-object 

Sentence Structure: Subject -verb-object relationship 



UNIT VERB VOCABULARY POST TEST 



UNIT FINAL SENTENCE TEST 



Appendix A-2: Content of lie Unit with Testing Schedule 
Pretest UNIT Verb Vocabulary 

SET ONE 

STANDARD TRACK ACCELERATED TRACK 



Pretest, Set One Progress Pretest, Set Oiie Progress 



BOOK I 
Section 
A Jump 
B Eat 

C Jump-Eat, 
Part 1 




BOOK I 
OMITTED 


BOOK n 




BOOK n 


Section 




Section 


A Jump 


Repetition 


A Jump 


B Eat 


of 


B Eat 


C Jump-Eat, 


BOOK I 


C Jump-Eat, 


Part 1 




Part 1 


C Jump-Eat, 




C Jump-Eat, 


Part 2 




Part 2 


BOOK in 




BOOK m 


Section 






D Paint 






E Wash 




OMITTED 


F Paint -Wash, 






Part 1 






BOOK IV 




BOOK TV 


Section 




Section 


D Paint 


Repetition 


D Paint 


/ E Wash 


— of 


E Wash 


F Paint -Wash, 


BOOK m 


F Paint-Wash, 


■ Part 1 




Part 1 


r. :-Y- ■ Paint -Wash, 




F Paint-Wash, 






Part 2 


G J ump -Eat - Paint - 


G Jump-Eat -Paint - 


: Wash 




Wash 








SET ONE (Gont»d) 



STANDARD TRACK 


ACCELERATED TRACK 


Pretest, Subject-No\in 
Vocabulary* 


Pretest, Subject-Noun 
Vocabulary* 


Pretest, Subject -Verb 
Relationship 


Pretest, Subject -Verb 
Relationship 


BOOK V 
Section 

H Verb in Sentence 
I Subject in Sentence 


BOOK V 
Section 

H Verb in Sentence 
I Subject in Sentence 


BOOK VI 
Section 

J Subject and Verb 
in Sentence 
K Subject and Verb 
in Sentence 


BOOK VI 
Section 

J Subject and Verb 
in Sentence 
K Subject and Verb 
in Sentence 


Posttest, Subject -Verb 
Relationship 


Posttest, Subject-Verb 
Relationship 



*CMldren who achieved mastery continued in Set One, Those who 
did not received the Subject NoTih Progranci (outlined on the following 
page)^ This test TOs admihistered to^ l^ subjects prior to 

■;-Set One'r ^ 



;Ch 



SUBJECT NOUN PROGRAM 



Pretest, Subject Noun Vocabulary* 



BOOK I 


BOOK! 


BOOK I 


Section 


Section 




A Boy 


A Boy 




B Girl 


B Girl 


OMITTED 


C Boy-Girl 


C B oy-Girl 




BOOK n 


BOOKH 


BOOKH 


Section 


Section 




D Man 


D Man 




E Woman 


E Woman 


OMITTED 


F Man- Woman 


F Man- Woman 




G Baby 


G Baby 




H Boy-Girl-Man- 


H Boy-Girl-Man- 




Woman-Baby 


Woman-Baby 




BOOK ni 


BOOK m 


BOOK HI 


Section 




Section 


A Dog 


. ■■■ . ■' ' • ■ 


A , Dog 


B Cat 




B Cat 


C Fish 


OMITTED ; ; 


C Fish ■ 


D Horse 




D Horse 


E Rabbit 




E Rabbit 


F Bird 




F Bird 



Posttest, Subj e ct Noun Vocabulary 



Noim Prpgrani were divided into 



m 



1 and II only. 



) 



SET TWO 



STANDARD TRACK 



ACCELERATED TRACK 



BOOK I 
Section 
A Run 
B Sit 
C Run-Sit, 
Part 1 




w a, A 0 

BOOK I 
OMITTED 


BOOKn 




BOOK n 


Section 




Section 


A Run 


Repetition 


A Run 


B Sit 


— of 


B Sit 


C R\in-Sit, 


BOOK I 


C Run-Sit, 


Part 1 




Part 1 


C Run-Sit, 




C Run-Sit, 


Part 2 




Part 2 \ , 


BOOKin 




BOOK in 


Section 






D Stand 






- E Walk 




OMITTED 


F Stand-Walk, 






Part 1 






BOOK rv 




BOOK IV 


Section 




Section 


D Stand 


Repetition 


D Stand 


E Walk 


of 


E Walk 


F Stand- Walk, 


BOOK in 


F Stand -Walk, 


Part 1 




■ ■■ Part 1 . 


F Stand -Walk, 




F Stand-Walk, 


Part 2 




Part 2 


G R un-Sit -Stand - 


G R un-Sit -Stand - 


Walk 




Walk 'v,- 



i^osttest. Set Tw 



Pdsttest, Set Twp^^^^P 






M 
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SET TWO (Cont»d) 



BOOK V - -Review 
Section 

H Cumulative Verb 
Vocabulary 
I Sentence Structure 
J Sentence Structure 



BOOKV r -Review 
Section 

H Cumulative Verb 
Vocabulary 
I Sentence Structure 
J Sentence Structure 



SET THREE 



STANDARD TRACK 



ACCELERATED TRACK 



BOOK I 
Section 

A Drink 
B Sleep 
C Drink-Sleep, 
Part 1 


Jk . 


BOOK I 
OMITTED 


BOOK n 
Section 

A Drink 


Repetition 


BOOK n 
Section 

A Drink 


B Sleep 


of 


B Sleep 


C Drink-Sleep, 


BOOK I 


C Drink-Sleep, 


Part 1 

C Drink-Sleep, 
Part 2 




Part .1 

C Drink-Sleep, 
Part '2 


BOOK m 




BOOK ni 


Section 

D Write 
E Cook 
F Write -Cook, 
Part 1 




OMITTED 


BOOK TV 




BOOK rv 


Section 




Section 


D Write 


Repeti^bii 


^ D Write 


E Cook 




. E..'.Cook ^ ^ ^ 


F Write-Cbok, 


BOOK III 


■ F Write-Cook, 


Part 1 




Part 1 


F Write-Cook, 




F Write-Cook, 


■ ■ xr; 




Part 2 


G Drink 


Write-. 


G Drink-Sleep-Write- 






Cook ■■ ■ 



Posttest, r Set Three 'Prdgr e^ 



'Posttest,; 



er|c 



T6 



SET THREE (Cont*d) 

STANDARD TRACK ACCELERATED TRACK 



BOOK V — Review BOOK V — Review 

Section Section 

H Cumulative Verb H Cumulative Verb 

Vocabxilary Vocabulary 

I Sentence Structure I Sentence Structure 

J Sentence Structure J Sentence Structure 



Because of problems in obtaining printed materials (see Appendix F), 
the remainder of the program could not be validated within the time 
limits of the project. Validation posttesting was conducted after Book V 
of Set Three. 



< 




BOOK VI BOOK VI 

Section Section 

K Verb-Object K Verb -Object ' 

Relationship Relationship 

Xj Vejrb-Object . - : Verb-Object 

Relationship Relationship 

M Verb-Object M Verb-Object 

Relationship Relationship 



tost continued in Set Three. 

Those who jEaiied receiyed^^^^^^^^^^ 
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SET FOUR 



STANDARD TRACK 



ACCELERATED TRACK 



Pr etest, Set Four Progress Pretest, Set Foux Progress 

BOOK I BOOK I 

Section 
A Draw 

B Cut OMITTED 

C Draw-Cut, 

Part 1 



BOOK n 

Section 

A Draw 
B Cut 

C Draw-Cut, 
Part 1 



Repetition 
— of 
BOOK I 



BOOK n 
Section 
A Draw 
B Cut 

C Draw -Cut, 
Part 1 



C Draw-Cut, 


C Draw-Cut, 


Part 2 


Part 2 



BOOKin 
Section 
D Push 
E Carry 
F Push-Carry, 
Part 1 




BOOKin 

OMITTED 


BOOK IV 




BOOK IV 


Section 




Section 


D Push 


Repetition 


D Push 


E Carry 


of 


E Carry 


F Push-Carry, 


BOOK m 


F Push-Carry, 


Part 1 




Part 1 


F Push-Carry, 




F Push-Carry, 


Part 2 




Part 2 


G Draw -Cut - Push - 


G Draw- Cut -Push- 


:■ ■'Carry"; ' 




Carry 



I^sttest, Set Four I^^ogr ess ; 



Posttest, Set Four ^Progress 



SET FOUR (Cont‘d) 



STANDARD TRACK 


ACCELERATED TRACK 


BOOK V--Review 


BOOK V- -Review 


Section 

H Cumulative Verb 
V ocabulary 

I Sentence Structure 
J Sentence Structure 


Section 

H Cumulative Verb 
V ocabxilary 
I Sentence Structure 
J Sentence Structure 


• 


BOOK VI 
Section 

K Subject, Verb and 

Object in Sentence 
L Subject, Verb and 

Object in Sentence 
M Subject, Verb and 

Object in Sentence 


BOOK VI 
Section 

K Subject, Verb and 

Object in Sentence 
L Subject, Verb and 

Object in Sentence 
M Subject, Verb and 

Object in Sentence 



Posttest UNIT Verb Vocabxilary (All Subjects) 



Final UNIT Sentence Test (All Subjects) 



Appendix B: Excerpts from the Program 



Appendix B contains excerpts from Set Three of the Program, Using Verbs , 
A Programmed Course for Deaf Children. It follows one verb through the 
teaching sequence. Like the other Sets, Set Three teaches four verbs divided 
into two teaching pairs as follows: 



First pair 

*A. First verb alone 
B. Second verb alone 
*C. First pair combined 
Second pair 

D. Third verb alone 
E* Fourth verb alone 
F. Second pair combined 



Four words of the set combined 
*G. 

R eview 

*H. Cumulative vocabulary review 
*1. Sentence Review 
*J. Sentence Review 



































. ■ • m 

CONCEPT DEVELOPMENT (A-1— A-9) 



♦Sections included in the excerpt 
S = Stimulus 

R = Response (correct choice) 

Alt. = Alternative (incorrect choice) 



All frames are photor educed. 
The actual frame size is 8-1/2 x 
11 ". 



Section A (23 frames) includes concept 
development and association of the 
printed word form for the first verb in 
the Set. 






80 

90 



' 1- 

^ >w«ru ^ > 


! 








J 

♦ 


1 

t 

* 


* 










S and R more dissimilar; R and Alt. 
more similar 



Association of mediating symbol 














PTr , „ 

















S and R very dissimilar; Alt. competi- 
tive 



Association of symbol; direct re* 




44 













\sf~’ 













’ 






' 















S and R very dissimilar; Alts. M^y Terminal frame for concept 
competitive _ _ 



ERIC 
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■ I 



drink 







A-13 



drink 



drink 




Done 




drift 


■ 


■ 







WORD RECOGNITION (A- 10— A- 13) 
Word matcMng with pictures 



Word matching; words competitive in 
form 




*-u 




A-W • 



print 



draw 



drink 






Word matching; (must attend word); 
Alt, competitive 



TER MINAL SEQUENCE (A- 14 - - A-2 3 ) 

Transition to picture terminal 



MZ: 








ou 


< 









_! 



drink 



I 






'racing and' copying frame (erasable 

9'i 



First picture t e r minal 



o 

ERIC 
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Traasition to word terminal with 




Picture terminal 






83 




83Er|c 



Picture terminal 






M3 





Word terininal; ^ts . highly competi 



tive 



ERJC Qd 




In the actual program a similar 
sequence for the verb sleep 
(Section B) follows Section A. 





4 



I 



I 

I 



i 

n 

3 

I 



if 

i 

I 



I 

'i 



i 



O 

me 







Section C brings the two words from 
the first teaching pair (drink and sleep ) 
together in a carefully controlled 
sequence. The words are alternated 
without the possibility of a response 
to the other word from the teaching 
pair. The two words appear in com- 
petition first in cued frames and then 
in terminals. 



sleep 



C-1 




Cued picture response for second 
wor^i^teachin^pair^drin^^o^gre^ent 




C-2 








Previous fremae type repeated for 

— 




o 

ERIC 
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sleep 




shoot 




0^ 

sprinkle 




■ M i ". . 


■ 



Cued word response for second word; 
drink not present 




n 

knock 




pour 




drink 


■ 


■ 


■ 



Previous frame type repeated for 





Tracing and copying frame for both 



words of pair 


















Sr 






















C-9 



meet 




sleep 




slide 



Picture response for sleep; drink not 
gresent^cue^^rogge^^^^^^^^^^^^ 



Frame type for C-7 repeated for 




c-7 



drink 



grin 



dance 




SlTord response for drink ; sleep not 
aresent; cues dropped ^ 



Copying frame; child must choose word 
*Red cues 




C-8- 








(SZ^ 














_j=^ JL j 






c-u 



sleep 



drink 






Trame type for C-6 repeated for drink : 



Copying frame; red cues dropped 






o 

ERIC 
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TERMINAL SEQUENCE: 

Part 2 of Section C 

From this point the two words 
from the teaching pair appear in 
competition. Accelerated students 
advance to Part 2. Standard track 
students repeat Sections A, B, and 
C-Part 1; then advance to Part 2, 




MZ] 




C-14 





Previous frame type repeated for 
sleep 




c-s 






1 

* H \ 

1 




sleep 


1 

■ 

1 ' 


drink 




*ued word response for drink 



Frame type for C-12 repeated for 





C-13 I 

■ 

' 1 • 




drink 


1 

1 


drink 



C-IS 











lued picture response for drink 



Picture terminal for drink 



ERIC 
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C-I7‘ 



sleep 



drink 




Word terminal for sleep 



C-tt 




Word terminal for drinlc 



C-19 



sleep 




Picture terminal for sleep 



In the actual program a similar 
sequence for the second teaching pair 
(write and cook) follows Section C. 

D - write separately 

E - cook separately 

F - write and cook combined 
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Section G brings the four words from the entire Set (drink, sleep, write 
and cook) together in a carefully controlled sequence. The words are alter- 
nated first without the possibility of a response to the other words from the 
Set. The four words appear in competition first in cued frames and then in 
terminals. 

Frames G-34 - G-45 teach recognition of the root word in a sentence 
with present progressive tense. 





Cued picture response for sleep ; other 




Previous frame type repeated for 
write ■ ' ' 






89 





Cued word response for drink; other I Previous frame type repeated for 
words from Set not present I cook 




Previous frame type repeated for write | Copying frame; child must choose word 




Previous frame type repeated for sleep i Picture response for write ; other wordi 

erJc , 100 






sleep 


G-11 




Ji 


0T 








'I vV 


































M4 






mock 



cook 



chant 



Previous frame type repeated for sleep 



Word response for cook ; other words 
from the Set not present 



cook 


&-12 




\-^7/ 









fi-B 





write 




weld 




smite 1 



Previous frame type repeated for cook 



Previous frame type repeated for write 



0 

drink 


6-23 



























Previous frame type repeated for drink 



e-16 



dive 




drink 




droop 


■ 


■ 


■ 



Previous frame type repeated for drink | 
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'revious frame type repeated 



TERMINAL, SEQUENCE: 




sleep 






Previous frame type repeated for write 



cook 



From this point the four words in 
the Set appear in competition - first 
ill cued frames and then in terminal 
frames. 






Previous frame type repeated for cook 



6-18 













drink 






sleep 


1 



»1 










:ued picture response for drink; other 












Pr e vi rms frame type repeated for sleep 



ERIC 



102 



92 





C-8 













write 




sleep 




drink 


■ 


■ 


A 


■ 



!ued word response for write ; otiber 

U3 








/6ifcT 




<JV 


sleep 




cook 




write 




■ 


B 



Previous frame type repeated for cook 

















cook 

. c 




<kkdc 




deep 



Previous frame type repeated for sleep 



ERIC 
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93 

















write 




drink 




cook 




Previoizs frame type repeated for drink 




Picture terminal for sleep 



write 



Mr 




Previous frame type repeated for write 



93 o 
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fr48t 







MI 






Previous frame type repeated for drink 






cook 

















* 








A 


/ — 




5 ^^ 







Previous frame type repeated for cook 




MO 



drink 



Word terminal for drink 



sleep 



cook 



write 



Previous frame type repeated for sleep 



6 ^ 



drink 



sleep 



cook 



Previous frame type repeated for cook 




6 ^ 



cook 



write 



drink 



Previous frame type repeated for write 



ERIC 
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M4 



The girl is drinlci ng. 



sleep 



cook 



drink 



e-37 



The girl is writi ng. 



drink 



write 



sleep 



WORD IN SENTENCE: (G-34— G-45) 
A.ttention to verb in sentence 



Previous frame type repeated for write 



G-3S 



U* 



The girl is sleeping. 



The girl is 






write 




sleep 


1 




■ 




s 



C 



cook . ! 



Previous frame type repeated for sleep 



G.36 



The girl » cooking. 



cook 




drink 




write 



1 

sfe^ng 


1 


cooking 




drinking 


■ ■ 


■ 



Cued verb with ing (root underscored) 



S-39 



The girl is 



cook ing 





drinking 




writing 



Previous frame type repeated for cook 



Previous frame type repeated for drink 



ERIC 
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■ ■: ■ 

Terminal for drink ; cues and under 
scoring removed 





Previous frame type repeated for cook 




G-44 



drinking 




sleeping 




writing 




Previous frame type repeated for write 




cooking 




sleeping 




drinking 


■ 


m 


■ 



Previous frame type repeated for sleep 



96 o 
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Section H is a review of the cumulative vocabulary as follows: 

Set Two (H-i - H-5) 

Set One and Set Two together (H-6 - H-14) 

Set One, Set Two and Set Three together (H-15 - H-27) 

Since Set One is reviewed in the latter sections of Set Two, it is reviewed less 
thoroughly than Set Two in the present Set, 



} 


H-1 


/ 

i 




stand 






run 






REVIEW - SET TWO: 

ctig^^esgons^^or^Jgjj^ 















Previous frame type repeated for run 
(running water) 




IM 






Previous frame type repeated for^ Previous frame type repeated for watt 



1 



o 

ERIC 
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run 


»-5 








, 




























■ 














eat 




run 




jump 



Previous frame type repeated for run 



Previous frame type repeated for run 
(Set Two) 




H-6 







A 














1. 1 


\ 






wash 




paint 




eat 






jump 1 


stand 




run 



IREVIEW - SETS ONE AND TWO: 

word response for wash (Set One) 



Previous frame type repeated for iiunp 
(Set One) 



-4 

1 


li-7 




A 


m 


















K-IO 





■a 

walk 



Previous frame type repeated for sit 



Previous frame type repeated for walk 
(Set Two) 



ERIC 
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H-U 






Previous frame type repeated for eat 
(Set One) 




H-14 




wash 




Previous frame type repeated for paint 
(Set One) 




H-12 




H-IS 









1 


1 ^ 










run 




A. 

walk 


1 

1 


1 

I stand 






jump 




write 




stand 



Previous frame type repeated for run 

ISsL 



REVIEW - SETS ONE, TWO & THREE: 
Wor^^ernuna^fo^writ^^Se^^liree^^^ 




H-13 




H-16 




stand 



cook 



jump 



Previous frame type repeated for standT Previous frame type for jump (Set One) 
(Set Two) M 



o 

ERIC 
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H-27 




Previous frame type for walk (Set Two) 




H-18 



wash 




stand 


1 drink 








L- 




Previous frame type for stand (Set Two] 




H-19 




Previous frame type for sleep (Set 




) Previous frame type for paint (Set One) 




run 




drink 




jump 


B 


B 


B 



)) Previous frame type for rxxn (Set Two) 




Previous frame type for drink (Set 
Three) ^ 



ion ^ 
ERIC 




cook 




run 




wash 



sit 




jump 




write 




Previous frame 




type for cook (Set 




Previous frame 




type for sit (Set Two) 







sleep 




eat 




run 



I Previous frame type for run (Set Two) 
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Section I reviews the picture response portion of sentence structure- -the 
relationship of the verb to the subject. Various devices .^re used to induce 
the student to attend both the subject and the verb and to integrate them. Vocab- 
ulary from all three Sets is included. 



The fish is eating. 



i-i 











Nlust attend subject only* Alts, constant 











The ck)g is running. 






The woman is sleeping 






The man is painting. 



1 - 3 ' 






ly^ust integrate subject and verb 



M 






Must attend verb only; Alts, constant 
except for verb — 



Previous frame type repeated 
■*SyTy»bols are red for attention_ 



ERIC 



11202 



© 1 






© A 


The hoy is standing. 






The girl b washing. 





Previous frame type with, red 
dropped 



The gin is drinking milk. 




print 




Frame type 1-6 repeated 



1-6 



The man is running. 



* 





1-9 








Previous frame type 



Frame type 1-7 repeated 
*Red underscore for attention 









The rabbit b jumping. 


* 




The woman b cooking. 



1-10 



* 




Must integrate subject and verb 
*Red underscore for attention 




Previous frame type 

,*Red underscore for attention 
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1-11 







UI4 








■ ■ ■ 




■ m 


■ 


Terminal frame; must integrate subject 
and verb with out cues 


Terminal frame 


t-12 






1 




The woman is drinking. 




The girl is jumping. 




... ri] 






- 







Terminal frame 



Terminal frame 




|.I3 









Terminal frame 



Terminal frame 



ERIC 



1-16 
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Section J reviews the word response portion of sentence structure--the 
relationship of the verb to the subject. Various devices are used to induce the 
student to attend to the subject in the sentence, locate it in the picture and 
select the appropriate verb. Vocabulary from all three Sets is included. 





FUNCTION OF ARROWS (J-1 - J-4) 
Must attend arrow to respond correctly 



J-2 





Previous frame type repeated; place 
ment of subject in picture shifted 






Previous frame type; placement of 
subiect in "Dicture shifted 





Previous frame type with word added 



ERIC 
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J-S 



The girl is. 



washing 




painting 




cooking 


■ 


■ 




■ 



Must locate subject and select verb; 
symbols to facilitate reading task 



The woman is 




J-6 






i 

Sitting 




«u- 

jumpino 




i 

standing 



Previous frame type repeated; place- 
ment of subject in picture shifted 




J-7 



The cat is 



drinking 




sleeping 




walking 



Previous frame type with response 



. 



er|c 



10 < 





running 




cooking 




jumping 


m 


■ 


■ 



Previous frame type; placement of 
subject in picture shifted 














eating 




writing 




painting 




Must attend subject and select verb 
*Red underscore for attention 



'fp 

The' horse is 




J-10 



.'it 










jumping 

i 


1 


sleeping 




drinking 






Previous frame type; red underscore 

I dropped 



ERIC 




J-ll 



& 

The woman is, 



sleeping 




woshing 




eating 



Previous type, response cues dropped 




J-12 



The woman is 



walking 




writing 




sitting 



Previous frame type; red underscore 




J-13 



The baby is. 



standing 




? 

running | 


painting 


■ 




■ 


■ ' 



Previous frame type repeated; place- 
Tnf*-nt of a subiect in pictu re shifted 





10*3 





J-I4 



The girl is. 



writing 




drinking 




walking 


■ 


■ 


■ 



Must read subject and select verb 
=^Red underscore for attention 

J-15 




The woman is 



cooking 





il 








sitting 




eating 



Previous frame type; red underscore 

J-I6 




The man is. 





» 










1 

<Q j 




painting 




sleeping I 


washing 


■ 




■ 


■ 



Previous frame type; placement of 
^ubjec^i^£ictur^_sb^ted^^^^__^ 
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J-17 



walking 




drinking 




running 


■ 


. 


■ 




■ 



Terminal frame; all cues dropped 




cooking 




eating 




sitting 


B 


B 


B 



Terminal frame 




painting 




jumping 


1 

1 writing 


B 


B E 



Terminal frame 
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Terminal frame 




J-21 




jumjairiq 



walking 




drinking 







Terminal frame 



108 
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Appendix C-1: Vocabulary Test Items for One Verb 

~9 






54 



walk 

wash 

sleep 

carry 




Appendix C-2: Vocabulary Test Items for Two Nouns 





woman 

room 

mouse 

girl 




rat 

bird 

bus 

dog 





110 



Appendix C-3: Sentence Structure Test T 






t Items 





Appendix D: Initial Behavior Testing-Training Sequence--Verbs, Day I 
SECTION A--Pro,cedure: 



The instructor performs an action while the child watches. 
A page containing verb pictures is exposed. 

The child selects his response from these pictures. 



(The following pictures are photoreduced. Actual pages are 8-1/2" x 11". ) 



Instructor jumps once acrbss the floor 



Instructor pulls a pull-toy. 



A-1. 



A-2. 







Instructor sweeps with a broom simi- 
lar to the one pictured and in a similar 
manner. 



Instructor repeats the action. 



A-3. 



A-4. 












L- 



i 
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Instructor tears a small piece of cloth. 



A-5. 












Instructor pours cereal from a box 
into a bowl. 



. A-6. 






SECTION B- -Procedure: 

The child looks at a picture of an action (left). 

He performs the action. 

Necessary props are available C-i a mat (right). 

Unnecessary props are included to prevent a response solely 
on the basis of the props. 












(Used props are removed, usually 
paper cup with water, ) 







O 

ERIC 
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(Used props are removed, usually 




(Props remaining from B-5. ) 



V y 



(No props are necessary for B~7 or B-8, ) 
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SECTION C--Procedure: 



The instructor covers the response pictures (lower portion of page) 
with a shield. 

The child looks at the stimulus picture (top of page). 

The shield la removed and the child selects his response. 





C-2. 




C-4. 
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Appendix E: Verb Use Test for the Deaf 



In addition to the Program validation procedure, it was originally- 
intended to compare the learning by the Program subjects with that of 
a control group. In order to obtain data for measuring the progress of 
the Program and the control subjects, the Verb Use Test for the Deaf 
was designed. Due to uncontrollable delays which occurred in the 
printing of conductive and non- conductive ink patches required for 
indicating Program responses, this aspect of the evaluation had to be 
abandoned. These problems are explained in Appendix F. 

An outline of the developmental stages of the test is presented 
below, followed by standardization data obtained on 506 children from 
eight schools for the deaf. 



TEST DEVELOPMENT 



I. Selection of content 

A. Vocabulary 

1. Sources of vocabulary 

a. CID Language Outline (first three levels) 

b. Index- -Instant Words (Words most used in reading and 
writing English) edited by Edward B. Fry, from word 
counts by R ins land, Thorndike, Lorge, Horne, Fitz- 
patrick and Dolch 

c. School materials 

(1) Reading texts 

(2) Workbooks 

2. Vocabulary usage 

a. Multiple meanings of one verb 

(1) . Selection of words 

(2) Selection of meanings for each word 

b. Multiple language for one situation 

(1) Selection of situations 

(2) Selection of language for each situation 

B. Tense 

1 . Importance in communication 

2. Difficulty of construction 

3. Difficulty of time concept involved 



II. Construction of items 

A. General reqtiirements 

1. Situation wi-thin experience of children 

2. Simple accompanying vocabulary 

3. Visual clue relation subject in sentence to illustration 



B. Vocabulary items 

1. Expressive 

a. niustrations 

(1) Single pictures 

(2) Sequence of pictures 

(3) Sequence with conversatiou balloons 

b. Elliptical sentence with multiple choice response 

2. Receptive-parallel items for selected expressive items 

C, Tense items 

1. Sequence with conversation balloons 

2, Time concept from illustration rather than "when" words 

III, Revision procedure 

A, First field testing 
1. Subjects 

a. Deaf subjects 

(1) Tennessee School for the Deaf--42 children from 
Eevels II, III and IV 

(2) Birmingham Day Program- -ten children from 
Levels I, II and III 

b. Hearing subjects 

(1) Two first-grade classes 

(2) Two seventh-grade classes 

(3) Selected children between third and seventh grade 
levels 

2„ Data obtained , 

a. Item analysis 

(1) Item difficulty (proportion answering each item 
correctly) 

(2) Proportion choosing each incorrect aiternative 

(3) Discriminating power (correlation of item with 
total score) 

b. Reliability 



Table I. Summary of Reliability Coefficient 

According to Kuder-Richardson Form- 
ula 20 for Types of Test Items 



Type of Test 


r Coefficient 


Expressive 




Part 1 


0. 967 


Part 2 


0. 968 


Receptive 


0. 832 


Tense 


0.S17 
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3, Editing and reassembling of items 

a. Revision of faulty items 

b. Elimination of ineffective items 

c. Addition of new items 

d. Elimination of receptive sections to reduce receptive- 
expressive redundancy and shorten test length 

e. Arranging items from easy to difficult 

B, Second field testing 
1, Subjects (deaf) 

a, Tennessee School for the Deaf--50 children from 
Bevels II through VIII 

b. Tampa---nlne children from Levels II through VIII 
E, Data obtained 

a. Item anally sis 

(1) Item difficulty 

(E) Proportion choosing each incorrect alternative 
(3) Discriminating power 

b. Reliability 

Table II. Summary of Reliability Coefficient 

According to Kuder -Richardson Form- 
ula EO for Types of Test Items on 
Second Field Testing 



Type of Test 


r Coefficient 


^ _ 1 ~ ~ ~ 

V ocabulary 




Items 1-100 


0. 967 


Items 100-200 


0. 968 


Tense 


0. 921 



C. Division of Vocabulary items into alternate test forms --A and B 

1, Equalization of the following 

a. Multiple meaning for the same word 

b. Variation of language for same situation 

c. Necessity of comprehending entire sentence 

d. Special constructions 

e. Lexical categories 

2, Statistical considerations 
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Table III. Summary of the Statistical Bases for Division of 

Vocabulary Items into Alternate Forms for Section I 
(single pictures), Section II (sequences of pictures), 
and Section III (sequence plus conversation balloon). 



Section 


Form 


Numbe r 
of Items 


Difficulty 
Range Mean 


Mean r 

with Total Score 


I 


A 


50 


,97-. 17 .55 


.47 




B 


50 


.97-. 12 .55 


.46 


II 


A 


27 


.95-. 25 .45 


. 50 




B 


, 27 


,88-. 24 .46 


. 50 


III 


A 


23 


.75-. 14 .39 


.46 




B 


23 


.71-. 12 .39 


.48 



D. Division of Tense items into alternate test forms- -A and B. 

1, Equalization of the following 

a. Simple past 

b. Simple present 

c. Present progressive 

d. Future with will 

e. Future with present progressive of go plus infinitive 
f* Negative 

2. Statistical considerations 

Table IV, Summary of Statistical Bases for Division 
of Tense Items into Alternate Test Forms 



Form 


Number 


Difficulty 


Mean r 




of Items 


Range Mean 


■with Total Score 


A 


25 


.51-. 29 .41 


.44 


B 


25 


.57-. 22 .41 


.44 



IV, Standardization 

A, Subjects --beginning in Lievels HI through VIII 

1, Alabama School for the Deaf 

2, Atlanta Speech School 

3, Dade County Public School Program for the Deaf (Miami) 

4, Florida School for the Deaf 

5, Georgia School for the Deaf 

6, Lexington School for the Deaf 

7, Nashville Public School Program for the Deaf 

8, North Carolina School for the Deaf (Morganton) 
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B, Administration of alternate test forms 
1 s First testing 

a. Form A--sample one (approximately half the subjects) 

b. Form B"-sample two (remaining Bubjects) 

E, Second testing (one week later) 

a. Form B --sample one 

b. Form A--sample two 



Table V, Correlations between Vocabulary Scores , Tense Scores, 
and Total Scores of Forms A and B of the Verb Use Test 
for the Deaf. Samples drawn from children enrolled in 
eight schools for the deaf. 



Age 

Level* 




V ocabulary 
1 00 Items 


Tense 
E5 Items 


T otal 
125 Items 


8 


38 


. 91 


.75 


. 92 


9 


69 


.92 


.70 


.93 


10 


•88 


.97 


. 88 


.97 


11 


77 


. 96 


.78 


. 96 


12 


98 


.97 


.85 


. 97 


13 


96 


.96 


.78 


,95 


14 


40 


. 97 


.89 


.98 


*The test 


was 


administered in 


October, Children 


reached the age 



sl'iown between January 1 and December 31 of that year. 
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Appendix F: Special Problems 

Serious disruption of the project schedule resulted from the prob- 
lems encountered with the printing of conductive and non-conductive ink 
patches required for the Program presentation. The reader is reminded 
that a student using the Program looks at a stimulus at the top of a page, 
selects his answer from three choices across the lower portion of the 
page and then touches the tip of the reinforcing device to the ink patch 
under his choice. The ink patch under the correct choice is printed 
with conductive ink and completes a circuit in the device causing it to 
light. The patches under the incorrect choices are non-' conductive and 
therefore do not complete the circuit. In order to prevent student re- 
sponses based on cues from the ink patches per se, all patches must be 
identical in appearance. An illustration of the Program presentation 
appears in {2,73) Final Model of the Teaching Machine, 

When satisfactory samples of conductive ink patches had been 
printed and tested with the reinforcing device for consistency of re- 
sponse and durability. It appeared that the major problems associated 
with the method of presentation had been solved. However, it subse- 
quently proved to be impossible to match the non-conductive patches, 
printed with conventional printers' ink, to the solid patch of conductive 
ink. Uniformity in appearance was then achieved by printing the non- 
conductive patches with the special base used for conductive ink, with- 
out the conductive component. Program samples printed by this method 
were carefully checked for appearance and conductive properties, and 
aU problema appeared to be solved. Although the production schedule 
had been delayed to some extent, it still appeared possible to begin 
classroom use of Program materials according to the original field 
testing schedule. 

When the first Program books were delivered, it was discovered 
that many of the wrong-answer patches (those under incorrect choices) 
contained sufficient conductive properties to activate the reinforcing 
device--thus incorrect responses would be reinforced. Efforts to de- 
termine the source of this contamination were confounded by the fact that 
those patches were changing in conductive properties with exposure to 
air. Materials which had been thoroughly checked upon delivery were 
later discovered to contain contaminated wrong-answer patches. The 
research departments of two ink companies were engaged by the printer 
in the search for a non-conductive ink which would match the conductive 
ink in appearance and could be used in a printing press. Since the con- 
ductive ink has an unusually flat finish, this task was a difficult one 
and approximately six weeks of fruitless research followed. 

In the interim the project staff sought a solution which would per- 
mit the use of conventional ink for non-conductive patches. The method 
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eventually adopted involved the use of patterned ink patches which 
negated the visible differences in sheen between the two types of ink. 

The new design of the response patch did not immediately resolve all 
problems. The printing of a pattern fine enough to provide sufficient 
conductivity for the right-answer patches proved to be difficult be- 
cause of the thick consiptency of the special Ink. In addition, some 
degree of contamination occurred in small areas of the wrong-answer 
patches; however, the degree of conductivity did not increase with ex- 
posure to air. Through the printer’s persistent efforts to devise pro- 
duction methods w'hich would eliminate contamination, it was gradu- 
ally reduced in subsequent materials and virtually eliminated in the 
last books printed. 

Although the problems associated with the special printing re- 
quired for the Program were eventually resolved, the loss of time 
for classroom use of instructional materials seriously Interfered 
with evaluation plans, which included both the validation of instruc- 
tional materials and the comparison of Program subjects with a con- 
trol group. Validation of instruction is considered a part of program 
development per se. A sample which is representative of the target 
population must use the materials under target conditions and their 
learning must be described by means of pretest and posttest measures. 
The validation tests cover th (3 specific content of the instructional ma- 
terials, and students are expected to achieve mastery on these tests 
after they have completed the program. In the present project, vali- 
dation tests were administered only to students who used the Program. 

In addition to validation,, it was originally intended to compare 
achievement of Program subjects, in one state residential school and 
one private day school, with the achievement of control subjects. The 
contro] group consisted of students in the same levels at the same two 
schools during the previous school year. For this purpose an achieve- 
ment test with alternate forms was developed and standardized. 

The standardized achievement test was administered in September 
and May of the 1966-1967 school year to the children expected to be the 
control subjects. At the beginning of the experimental year (1967-1968) 
some delay in the production of materials had been encountered; however, 
it appeared that the original plans could still be executed on schedule. 

The standardized achievement test was administered to the state school 
children in L,evel8 II, III and IV, It was expected that these children 
would constitute the major portion of the experimental group. Prelimi- 
nary Program activities, including validation pretesting and the Initial 
Behavior Testing-Training Sequence, were also administered to these 
children in preparation for beginning the Program, The first Program 
books were delivered the evening before the designated time for their 
use and the schedule appeared to be going according to plan. It was then 



discovered that wrong-answer patches were contaminated- -these 
faulty materials could not be introduced to the students. 

When acceptable materials were finally obtained, more than for r 
m.onths had elapsed; validation pretesting could no longer b(' considered 
valid and the effectiveness of IBT-TS was questionable. Project con- 
sultants advised against repeating these measures with the same chil- 
dren. It was apparent that the Program sample for which control data 
were available must be abandoned and that evaluations based on direct 
comparisons of control and experimental samples would not be possible. 

It appeared that validation of the Program, considered the crucial 
aspect of the evaluation, would still be feasible; therefore a new sample 
was sought for thi|s purpose. Since student use of the Program had been 
delayed from September lantil January, it was possible for students in 
their first year of school to participate. However, the number of 
Level I children in the two- participating schools was not sufficient for 
the purpose of validation. The cooperation ojf another state school made 
it possible to obtain an adequate number of sbbjects and also to include 
children in Level IL 

All problems with printing had not been resolved at the time the 
Program was introduced to the students. In spite of persistent efforts 
on the part of the printer to overcome production difficulties, delays 
continued to occur throughout the validation period. The last Set of 
the Program, consisting of six books, and one book of the previous 
Set could not be produced early enough in the school year to be used 
by the validation sample. The most unfortunate result of these delays 
was the loss of time necessai'y to vaTidate these final portions of the 
Program- -approximately 25% of the total Program Unit. 
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A Si TRAC T 



the project was to develop and field test a program of 
iiwtruction for young deaf children which could be used in existing classrooms. 
The main Program contained two areas of instruction in written language --verb 
vocabulary at a high level of generalization and sent jnce structure. The Pro- 
gram materials were field tested with 78 children in the preparatory levels of 
three schools. The only criterion for admission to the sample was a simple test 
of minimum word recognition ability. Subjects ranged from five to ten years of 

age, and from three and one-half to ten and one-half years on psychometric 
scores. 



Three validation criteria--one final performance criterion and two measures 
of gam— were applied in 28 instances to the pre-post-test data from the sample 
groups. The criteria were met in 21 instances. Of the 78 children, 77% 
achieved mastery in verb vocabulary, and 83% in sentences. Each of the sample 
groups evidenced statistically significant learning in all areas of instruction, 
pie mean time required for the Program was 4. 7 hours. It was concluded that 
the Ppgram represents a very effective, as well as an efficient, method of 
teaching written language to young deaf children. 






